2022-01-30 22:47:39,297 INFO    MainThread:23196 [internal.py:wandb_internal():87] W&B internal server running at pid: 23196, started at: 2022-01-30 22:47:39.296970
2022-01-30 22:47:39,300 INFO    WriterThread:23196 [datastore.py:open_for_write():77] open: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/run-2uzt3kt1.wandb
2022-01-30 22:47:39,301 DEBUG   HandlerThread:23196 [handler.py:handle_request():130] handle_request: check_version
2022-01-30 22:47:39,304 DEBUG   SenderThread:23196 [sender.py:send():234] send: header
2022-01-30 22:47:39,304 DEBUG   SenderThread:23196 [sender.py:send_request():248] send_request: check_version
2022-01-30 22:47:39,377 DEBUG   SenderThread:23196 [sender.py:send():234] send: run
2022-01-30 22:47:39,597 DEBUG   HandlerThread:23196 [handler.py:handle_request():130] handle_request: run_start
2022-01-30 22:47:39,599 INFO    SenderThread:23196 [dir_watcher.py:__init__():169] watching files in: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files
2022-01-30 22:47:39,600 INFO    SenderThread:23196 [sender.py:_start_run_threads():804] run started: 2uzt3kt1 with start time 1643582858
2022-01-30 22:47:39,600 DEBUG   SenderThread:23196 [sender.py:send():234] send: summary
2022-01-30 22:47:39,600 INFO    SenderThread:23196 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
2022-01-30 22:47:39,606 DEBUG   HandlerThread:23196 [meta.py:__init__():40] meta init
2022-01-30 22:47:39,606 DEBUG   HandlerThread:23196 [meta.py:__init__():54] meta init done
2022-01-30 22:47:39,606 DEBUG   HandlerThread:23196 [meta.py:probe():214] probe
2022-01-30 22:47:39,615 DEBUG   HandlerThread:23196 [meta.py:_setup_git():204] setup git
2022-01-30 22:47:39,653 DEBUG   HandlerThread:23196 [meta.py:_setup_git():211] setup git done
2022-01-30 22:47:39,653 DEBUG   HandlerThread:23196 [meta.py:_save_pip():58] save pip
2022-01-30 22:47:39,654 DEBUG   HandlerThread:23196 [meta.py:_save_pip():72] save pip done
2022-01-30 22:47:39,655 DEBUG   HandlerThread:23196 [meta.py:_save_conda():79] save conda
2022-01-30 22:47:40,176 DEBUG   HandlerThread:23196 [meta.py:_save_conda():89] save conda done
2022-01-30 22:47:40,176 DEBUG   HandlerThread:23196 [meta.py:probe():252] probe done
2022-01-30 22:47:40,185 DEBUG   SenderThread:23196 [sender.py:send():234] send: files
2022-01-30 22:47:40,186 INFO    SenderThread:23196 [sender.py:_save_file():939] saving file wandb-metadata.json with policy now
2022-01-30 22:47:40,197 DEBUG   HandlerThread:23196 [handler.py:handle_request():130] handle_request: stop_status
2022-01-30 22:47:40,198 DEBUG   SenderThread:23196 [sender.py:send_request():248] send_request: stop_status
2022-01-30 22:47:40,354 DEBUG   SenderThread:23196 [sender.py:send():234] send: config
2022-01-30 22:47:40,357 DEBUG   SenderThread:23196 [sender.py:send():234] send: metric
2022-01-30 22:47:40,357 DEBUG   SenderThread:23196 [sender.py:send():234] send: metric
2022-01-30 22:47:40,357 WARNING SenderThread:23196 [sender.py:send_metric():897] Seen metric with glob (shouldnt happen)
2022-01-30 22:47:40,601 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_created():217] file/dir created: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/requirements.txt
2022-01-30 22:47:40,601 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_created():217] file/dir created: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/wandb-summary.json
2022-01-30 22:47:40,601 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_created():217] file/dir created: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/output.log
2022-01-30 22:47:40,601 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_created():217] file/dir created: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/wandb-metadata.json
2022-01-30 22:47:40,601 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_created():217] file/dir created: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/conda-environment.yaml
2022-01-30 22:47:40,709 INFO    Thread-11 :23196 [upload_job.py:push():137] Uploaded file /tmp/tmp51rrl_mrwandb/2urk4c1m-wandb-metadata.json
2022-01-30 22:47:42,601 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/output.log
2022-01-30 22:47:46,604 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/output.log
2022-01-30 22:47:50,606 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/output.log
2022-01-30 22:47:50,676 DEBUG   SenderThread:23196 [sender.py:send():234] send: metric
2022-01-30 22:47:50,677 DEBUG   SenderThread:23196 [sender.py:send():234] send: metric
2022-01-30 22:47:50,677 DEBUG   SenderThread:23196 [sender.py:send():234] send: metric
2022-01-30 22:47:50,677 DEBUG   SenderThread:23196 [sender.py:send():234] send: metric
2022-01-30 22:47:50,677 DEBUG   SenderThread:23196 [sender.py:send():234] send: metric
2022-01-30 22:47:50,677 DEBUG   SenderThread:23196 [sender.py:send():234] send: metric
2022-01-30 22:47:50,678 DEBUG   SenderThread:23196 [sender.py:send():234] send: history
2022-01-30 22:47:50,678 DEBUG   SenderThread:23196 [sender.py:send():234] send: summary
2022-01-30 22:47:50,679 INFO    SenderThread:23196 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
2022-01-30 22:47:51,607 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/wandb-summary.json
2022-01-30 22:47:52,608 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/output.log
2022-01-30 22:47:54,610 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/output.log
2022-01-30 22:47:55,488 DEBUG   HandlerThread:23196 [handler.py:handle_request():130] handle_request: stop_status
2022-01-30 22:47:55,488 DEBUG   SenderThread:23196 [sender.py:send_request():248] send_request: stop_status
2022-01-30 22:47:56,611 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/output.log
2022-01-30 22:48:07,838 DEBUG   SenderThread:23196 [sender.py:send():234] send: stats
2022-01-30 22:48:10,621 INFO    Thread-8  :23196 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220130_224738-2uzt3kt1/files/config.yaml
2022-01-30 22:48:10,741 DEBUG   HandlerThread:23196 [handler.py:handle_request():130] handle_request: stop_status
2022-01-30 22:48:10,742 DEBUG   SenderThread:23196 [sender.py:send_request():248] send_request: stop_status