kertob commited on
Commit
d6a458f
1 Parent(s): 5ef9455

Training in progress, step 190

Browse files
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:12b4877e33932f546ab6653b795adaac9393810aa60a67d4726d7f7bd3810db2
3
  size 261131840
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5a5f27c3f1728826e5350b88b701dbaa95988bb2c2eb31b96d3ff4adf826b689
3
  size 261131840
runs/Mar08_13-48-53_4997a2709b0f/events.out.tfevents.1709905734.4997a2709b0f.35964.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a22aac4d8d26e14511d71ceeaa84255d5afbb1dfaeb19e1743ea13dd81f07b18
3
- size 9482
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:161deb8582296e0a898831e8e25742babba4c9eb989b99c479251d1b1ac3b303
3
+ size 9693
wandb/debug-internal.log CHANGED
@@ -875,3 +875,47 @@
875
  2024-03-08 14:17:37,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
876
  2024-03-08 14:17:37,820 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
877
  2024-03-08 14:17:43,099 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
875
  2024-03-08 14:17:37,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
876
  2024-03-08 14:17:37,820 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
877
  2024-03-08 14:17:43,099 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
878
+ 2024-03-08 14:17:48,100 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
879
+ 2024-03-08 14:17:52,819 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
880
+ 2024-03-08 14:17:52,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
881
+ 2024-03-08 14:17:52,821 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
882
+ 2024-03-08 14:17:55,450 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
883
+ 2024-03-08 14:18:00,451 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
884
+ 2024-03-08 14:18:05,452 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
885
+ 2024-03-08 14:18:07,216 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
886
+ 2024-03-08 14:18:07,819 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
887
+ 2024-03-08 14:18:07,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
888
+ 2024-03-08 14:18:07,821 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
889
+ 2024-03-08 14:18:11,096 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
890
+ 2024-03-08 14:18:16,098 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
891
+ 2024-03-08 14:18:21,099 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
892
+ 2024-03-08 14:18:22,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
893
+ 2024-03-08 14:18:22,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
894
+ 2024-03-08 14:18:22,821 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
895
+ 2024-03-08 14:18:27,066 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
896
+ 2024-03-08 14:18:32,067 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
897
+ 2024-03-08 14:18:37,068 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
898
+ 2024-03-08 14:18:37,217 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
899
+ 2024-03-08 14:18:37,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
900
+ 2024-03-08 14:18:37,821 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
901
+ 2024-03-08 14:18:37,821 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
902
+ 2024-03-08 14:18:42,093 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
903
+ 2024-03-08 14:18:47,094 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
904
+ 2024-03-08 14:18:50,476 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: partial_history
905
+ 2024-03-08 14:18:50,477 DEBUG SenderThread:37167 [sender.py:send():378] send: history
906
+ 2024-03-08 14:18:50,478 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: summary_record
907
+ 2024-03-08 14:18:50,478 INFO SenderThread:37167 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end
908
+ 2024-03-08 14:18:50,612 INFO Thread-12 :37167 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240308_134901-hlg2f18u/files/wandb-summary.json
909
+ 2024-03-08 14:18:52,479 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
910
+ 2024-03-08 14:18:52,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
911
+ 2024-03-08 14:18:52,821 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
912
+ 2024-03-08 14:18:52,821 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
913
+ 2024-03-08 14:18:57,526 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
914
+ 2024-03-08 14:19:02,527 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
915
+ 2024-03-08 14:19:07,217 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
916
+ 2024-03-08 14:19:07,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
917
+ 2024-03-08 14:19:07,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
918
+ 2024-03-08 14:19:07,821 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
919
+ 2024-03-08 14:19:07,822 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
920
+ 2024-03-08 14:19:13,138 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
921
+ 2024-03-08 14:19:18,139 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240308_134901-hlg2f18u/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 0.1517, "train/grad_norm": 0.5520470142364502, "train/learning_rate": 2.5995410021864787e-06, "train/epoch": 14.4, "train/global_step": 180, "_timestamp": 1709907437.5697315, "_runtime": 1696.1494345664978, "_step": 17}
 
1
+ {"train/loss": 0.1169, "train/grad_norm": 0.517312228679657, "train/learning_rate": 6.54164563305465e-07, "train/epoch": 15.2, "train/global_step": 190, "_timestamp": 1709907530.4761915, "_runtime": 1789.055894613266, "_step": 18}
wandb/run-20240308_134901-hlg2f18u/logs/debug-internal.log CHANGED
@@ -875,3 +875,47 @@
875
  2024-03-08 14:17:37,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
876
  2024-03-08 14:17:37,820 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
877
  2024-03-08 14:17:43,099 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
875
  2024-03-08 14:17:37,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
876
  2024-03-08 14:17:37,820 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
877
  2024-03-08 14:17:43,099 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
878
+ 2024-03-08 14:17:48,100 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
879
+ 2024-03-08 14:17:52,819 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
880
+ 2024-03-08 14:17:52,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
881
+ 2024-03-08 14:17:52,821 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
882
+ 2024-03-08 14:17:55,450 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
883
+ 2024-03-08 14:18:00,451 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
884
+ 2024-03-08 14:18:05,452 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
885
+ 2024-03-08 14:18:07,216 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
886
+ 2024-03-08 14:18:07,819 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
887
+ 2024-03-08 14:18:07,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
888
+ 2024-03-08 14:18:07,821 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
889
+ 2024-03-08 14:18:11,096 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
890
+ 2024-03-08 14:18:16,098 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
891
+ 2024-03-08 14:18:21,099 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
892
+ 2024-03-08 14:18:22,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
893
+ 2024-03-08 14:18:22,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
894
+ 2024-03-08 14:18:22,821 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
895
+ 2024-03-08 14:18:27,066 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
896
+ 2024-03-08 14:18:32,067 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
897
+ 2024-03-08 14:18:37,068 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
898
+ 2024-03-08 14:18:37,217 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
899
+ 2024-03-08 14:18:37,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
900
+ 2024-03-08 14:18:37,821 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
901
+ 2024-03-08 14:18:37,821 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
902
+ 2024-03-08 14:18:42,093 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
903
+ 2024-03-08 14:18:47,094 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
904
+ 2024-03-08 14:18:50,476 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: partial_history
905
+ 2024-03-08 14:18:50,477 DEBUG SenderThread:37167 [sender.py:send():378] send: history
906
+ 2024-03-08 14:18:50,478 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: summary_record
907
+ 2024-03-08 14:18:50,478 INFO SenderThread:37167 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end
908
+ 2024-03-08 14:18:50,612 INFO Thread-12 :37167 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240308_134901-hlg2f18u/files/wandb-summary.json
909
+ 2024-03-08 14:18:52,479 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
910
+ 2024-03-08 14:18:52,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
911
+ 2024-03-08 14:18:52,821 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
912
+ 2024-03-08 14:18:52,821 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
913
+ 2024-03-08 14:18:57,526 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
914
+ 2024-03-08 14:19:02,527 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
915
+ 2024-03-08 14:19:07,217 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
916
+ 2024-03-08 14:19:07,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
917
+ 2024-03-08 14:19:07,820 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
918
+ 2024-03-08 14:19:07,821 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
919
+ 2024-03-08 14:19:07,822 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
920
+ 2024-03-08 14:19:13,138 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
921
+ 2024-03-08 14:19:18,139 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240308_134901-hlg2f18u/run-hlg2f18u.wandb CHANGED
Binary files a/wandb/run-20240308_134901-hlg2f18u/run-hlg2f18u.wandb and b/wandb/run-20240308_134901-hlg2f18u/run-hlg2f18u.wandb differ