kertob commited on
Commit
1179e36
1 Parent(s): 10d9223

Training in progress, step 20

Browse files
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d3a90f573ff45ffb642261678ab6f137a94e4bdd6abd7143e513d6f48fb1d0ed
3
  size 261131840
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:07776e436d924c4c90a88d540337e60a8389219b6dec1254087965f2a51bb0a0
3
  size 261131840
runs/Mar08_13-48-53_4997a2709b0f/events.out.tfevents.1709905734.4997a2709b0f.35964.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fc645a3eb47788939617ac9f3e8aa940658aab707e99a387b4f2399e2f7d030e
3
- size 5939
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb8fd1f63f283a5f308302acebfcba57882fe0e0eb0abaf1e6811fd6c6903a10
3
+ size 6146
wandb/debug-internal.log CHANGED
@@ -104,3 +104,44 @@
104
  2024-03-08 13:51:01,937 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
105
  2024-03-08 13:51:06,939 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
106
  2024-03-08 13:51:07,171 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
104
  2024-03-08 13:51:01,937 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
105
  2024-03-08 13:51:06,939 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
106
  2024-03-08 13:51:07,171 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
107
+ 2024-03-08 13:51:07,648 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
108
+ 2024-03-08 13:51:07,648 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
109
+ 2024-03-08 13:51:07,689 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
110
+ 2024-03-08 13:51:11,951 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
111
+ 2024-03-08 13:51:16,952 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
112
+ 2024-03-08 13:51:21,953 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
113
+ 2024-03-08 13:51:22,648 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
114
+ 2024-03-08 13:51:22,649 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
115
+ 2024-03-08 13:51:22,689 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
116
+ 2024-03-08 13:51:27,772 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
117
+ 2024-03-08 13:51:32,773 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
118
+ 2024-03-08 13:51:37,172 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
119
+ 2024-03-08 13:51:37,650 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
120
+ 2024-03-08 13:51:37,650 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
121
+ 2024-03-08 13:51:37,651 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
122
+ 2024-03-08 13:51:37,939 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
123
+ 2024-03-08 13:51:42,940 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
124
+ 2024-03-08 13:51:47,942 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
125
+ 2024-03-08 13:51:52,649 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
126
+ 2024-03-08 13:51:52,649 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
127
+ 2024-03-08 13:51:52,689 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
128
+ 2024-03-08 13:51:54,693 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
129
+ 2024-03-08 13:51:59,617 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: partial_history
130
+ 2024-03-08 13:51:59,619 DEBUG SenderThread:37167 [sender.py:send():378] send: history
131
+ 2024-03-08 13:51:59,619 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: summary_record
132
+ 2024-03-08 13:51:59,621 INFO SenderThread:37167 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end
133
+ 2024-03-08 13:52:00,002 INFO Thread-12 :37167 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240308_134901-hlg2f18u/files/wandb-summary.json
134
+ 2024-03-08 13:52:00,659 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
135
+ 2024-03-08 13:52:01,002 INFO Thread-12 :37167 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240308_134901-hlg2f18u/files/output.log
136
+ 2024-03-08 13:52:05,660 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
137
+ 2024-03-08 13:52:07,173 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
138
+ 2024-03-08 13:52:07,649 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
139
+ 2024-03-08 13:52:07,650 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
140
+ 2024-03-08 13:52:07,651 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
141
+ 2024-03-08 13:52:10,920 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
142
+ 2024-03-08 13:52:15,921 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
143
+ 2024-03-08 13:52:20,922 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
144
+ 2024-03-08 13:52:22,649 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
145
+ 2024-03-08 13:52:22,650 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
146
+ 2024-03-08 13:52:22,690 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
147
+ 2024-03-08 13:52:26,913 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240308_134901-hlg2f18u/files/output.log CHANGED
@@ -1 +1,2 @@
1
 
 
 
1
 
2
+ Checkpoint destination directory ./checkpoint-10 already exists and is non-empty. Saving will proceed but saved results may be invalid.
wandb/run-20240308_134901-hlg2f18u/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 2.4942, "train/grad_norm": 0.6082926988601685, "train/learning_rate": 9.989514131188559e-05, "train/epoch": 0.8, "train/global_step": 10, "_timestamp": 1709905830.8509364, "_runtime": 89.43063950538635, "_step": 0}
 
1
+ {"train/loss": 1.2173, "train/grad_norm": 0.9394240379333496, "train/learning_rate": 9.872052623234632e-05, "train/epoch": 1.6, "train/global_step": 20, "_timestamp": 1709905919.6171632, "_runtime": 178.19686627388, "_step": 1}
wandb/run-20240308_134901-hlg2f18u/logs/debug-internal.log CHANGED
@@ -104,3 +104,44 @@
104
  2024-03-08 13:51:01,937 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
105
  2024-03-08 13:51:06,939 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
106
  2024-03-08 13:51:07,171 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
104
  2024-03-08 13:51:01,937 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
105
  2024-03-08 13:51:06,939 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
106
  2024-03-08 13:51:07,171 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
107
+ 2024-03-08 13:51:07,648 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
108
+ 2024-03-08 13:51:07,648 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
109
+ 2024-03-08 13:51:07,689 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
110
+ 2024-03-08 13:51:11,951 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
111
+ 2024-03-08 13:51:16,952 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
112
+ 2024-03-08 13:51:21,953 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
113
+ 2024-03-08 13:51:22,648 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
114
+ 2024-03-08 13:51:22,649 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
115
+ 2024-03-08 13:51:22,689 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
116
+ 2024-03-08 13:51:27,772 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
117
+ 2024-03-08 13:51:32,773 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
118
+ 2024-03-08 13:51:37,172 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
119
+ 2024-03-08 13:51:37,650 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
120
+ 2024-03-08 13:51:37,650 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
121
+ 2024-03-08 13:51:37,651 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
122
+ 2024-03-08 13:51:37,939 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
123
+ 2024-03-08 13:51:42,940 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
124
+ 2024-03-08 13:51:47,942 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
125
+ 2024-03-08 13:51:52,649 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
126
+ 2024-03-08 13:51:52,649 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
127
+ 2024-03-08 13:51:52,689 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
128
+ 2024-03-08 13:51:54,693 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
129
+ 2024-03-08 13:51:59,617 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: partial_history
130
+ 2024-03-08 13:51:59,619 DEBUG SenderThread:37167 [sender.py:send():378] send: history
131
+ 2024-03-08 13:51:59,619 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: summary_record
132
+ 2024-03-08 13:51:59,621 INFO SenderThread:37167 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end
133
+ 2024-03-08 13:52:00,002 INFO Thread-12 :37167 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240308_134901-hlg2f18u/files/wandb-summary.json
134
+ 2024-03-08 13:52:00,659 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
135
+ 2024-03-08 13:52:01,002 INFO Thread-12 :37167 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240308_134901-hlg2f18u/files/output.log
136
+ 2024-03-08 13:52:05,660 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
137
+ 2024-03-08 13:52:07,173 DEBUG SenderThread:37167 [sender.py:send():378] send: stats
138
+ 2024-03-08 13:52:07,649 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
139
+ 2024-03-08 13:52:07,650 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
140
+ 2024-03-08 13:52:07,651 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
141
+ 2024-03-08 13:52:10,920 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
142
+ 2024-03-08 13:52:15,921 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
143
+ 2024-03-08 13:52:20,922 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
144
+ 2024-03-08 13:52:22,649 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: stop_status
145
+ 2024-03-08 13:52:22,650 DEBUG SenderThread:37167 [sender.py:send_request():405] send_request: stop_status
146
+ 2024-03-08 13:52:22,690 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: internal_messages
147
+ 2024-03-08 13:52:26,913 DEBUG HandlerThread:37167 [handler.py:handle_request():146] handle_request: status_report
wandb/run-20240308_134901-hlg2f18u/run-hlg2f18u.wandb CHANGED
Binary files a/wandb/run-20240308_134901-hlg2f18u/run-hlg2f18u.wandb and b/wandb/run-20240308_134901-hlg2f18u/run-hlg2f18u.wandb differ