Hugging Face's logo

MattStammers
/
appo-mujoco_humanoid-sota

Reinforcement Learning
sample-factory
TensorBoard
deep-reinforcement-learning
Eval Results
Model card Files Metrics Community
MattStammers commited on
Commit
cca76a5
•
1 Parent(s): 585d552

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +2 -0
  2. .summary/0/events.out.tfevents.1703455886.rhmmedcatt-proliant-ml350-gen10 +3 -0
  3. .summary/1/events.out.tfevents.1703455886.rhmmedcatt-proliant-ml350-gen10 +3 -0
  4. README.md +189 -0
  5. checkpoint_p0/best_001912832_489684992_reward_11497.130.pth +3 -0
  6. checkpoint_p0/checkpoint_001953056_499982336.pth +3 -0
  7. checkpoint_p0/checkpoint_001953136_500006912.pth +3 -0
  8. checkpoint_p0/milestones/checkpoint_000027040_6922240.pth +3 -0
  9. checkpoint_p0/milestones/checkpoint_000054464_13942784.pth +3 -0
  10. checkpoint_p0/milestones/checkpoint_000081856_20955136.pth +3 -0
  11. checkpoint_p0/milestones/checkpoint_000109536_28041216.pth +3 -0
  12. checkpoint_p0/milestones/checkpoint_000136928_35053568.pth +3 -0
  13. checkpoint_p0/milestones/checkpoint_000164128_42016768.pth +3 -0
  14. checkpoint_p0/milestones/checkpoint_000191712_49078272.pth +3 -0
  15. checkpoint_p0/milestones/checkpoint_000219680_56238080.pth +3 -0
  16. checkpoint_p0/milestones/checkpoint_000247616_63389696.pth +3 -0
  17. checkpoint_p0/milestones/checkpoint_000275232_70459392.pth +3 -0
  18. checkpoint_p0/milestones/checkpoint_000302272_77381632.pth +3 -0
  19. checkpoint_p0/milestones/checkpoint_000329152_84262912.pth +3 -0
  20. checkpoint_p0/milestones/checkpoint_000355968_91127808.pth +3 -0
  21. checkpoint_p0/milestones/checkpoint_000382688_97968128.pth +3 -0
  22. checkpoint_p0/milestones/checkpoint_000409184_104751104.pth +3 -0
  23. checkpoint_p0/milestones/checkpoint_000435392_111460352.pth +3 -0
  24. checkpoint_p0/milestones/checkpoint_000461888_118243328.pth +3 -0
  25. checkpoint_p0/milestones/checkpoint_000488160_124968960.pth +3 -0
  26. checkpoint_p0/milestones/checkpoint_000514304_131661824.pth +3 -0
  27. checkpoint_p0/milestones/checkpoint_000540512_138371072.pth +3 -0
  28. checkpoint_p0/milestones/checkpoint_000566592_145047552.pth +3 -0
  29. checkpoint_p0/milestones/checkpoint_000592864_151773184.pth +3 -0
  30. checkpoint_p0/milestones/checkpoint_000618912_158441472.pth +3 -0
  31. checkpoint_p0/milestones/checkpoint_000644960_165109760.pth +3 -0
  32. checkpoint_p0/milestones/checkpoint_000670912_171753472.pth +3 -0
  33. checkpoint_p0/milestones/checkpoint_000696768_178372608.pth +3 -0
  34. checkpoint_p0/milestones/checkpoint_000722592_184983552.pth +3 -0
  35. checkpoint_p0/milestones/checkpoint_000748192_191537152.pth +3 -0
  36. checkpoint_p0/milestones/checkpoint_000773504_198017024.pth +3 -0
  37. checkpoint_p0/milestones/checkpoint_000799008_204546048.pth +3 -0
  38. checkpoint_p0/milestones/checkpoint_000824576_211091456.pth +3 -0
  39. checkpoint_p0/milestones/checkpoint_000850208_217653248.pth +3 -0
  40. checkpoint_p0/milestones/checkpoint_000875936_224239616.pth +3 -0
  41. checkpoint_p0/milestones/checkpoint_000901856_230875136.pth +3 -0
  42. checkpoint_p0/milestones/checkpoint_000927584_237461504.pth +3 -0
  43. checkpoint_p0/milestones/checkpoint_000953440_244080640.pth +3 -0
  44. checkpoint_p0/milestones/checkpoint_000979360_250716160.pth +3 -0
  45. checkpoint_p0/milestones/checkpoint_001005376_257376256.pth +3 -0
  46. checkpoint_p0/milestones/checkpoint_001031168_263979008.pth +3 -0
  47. checkpoint_p0/milestones/checkpoint_001056704_270516224.pth +3 -0
  48. checkpoint_p0/milestones/checkpoint_001082176_277037056.pth +3 -0
  49. checkpoint_p0/milestones/checkpoint_001107840_283607040.pth +3 -0
  50. checkpoint_p0/milestones/checkpoint_001133280_290119680.pth +3 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ replay.mp4 filter=lfs diff=lfs merge=lfs -text
37
+ sf_log.txt filter=lfs diff=lfs merge=lfs -text
.summary/0/events.out.tfevents.1703455886.rhmmedcatt-proliant-ml350-gen10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:105c2f944087a9c6eba3f062508d2baa3fd0654e2218e667bfe2985cd36b7875
3
+ size 39913827
.summary/1/events.out.tfevents.1703455886.rhmmedcatt-proliant-ml350-gen10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48fa7786e538caf1834aee88b20aa2283dd1b7c59a99e04e7da71a144b9c4e29
3
+ size 18802359
README.md ADDED
@@ -0,0 +1,189 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: sample-factory
3
+ tags:
4
+ - deep-reinforcement-learning
5
+ - reinforcement-learning
6
+ - sample-factory
7
+ model-index:
8
+ - name: APPO
9
+ results:
10
+ - task:
11
+ type: reinforcement-learning
12
+ name: reinforcement-learning
13
+ dataset:
14
+ name: mujoco_humanoid
15
+ type: mujoco_humanoid
16
+ metrics:
17
+ - type: mean_reward
18
+ value: 11486.26 +/- 29.38
19
+ name: mean_reward
20
+ verified: false
21
+ ---
22
+
23
+ ## About the Project
24
+
25
+ This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist).
26
+
27
+ In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
28
+
29
+ ## Project Aims
30
+
31
+ This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance.
32
+
33
+ I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota.
34
+
35
+ The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels.
36
+
37
+ After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
38
+
39
+
40
+ ## About the Model
41
+
42
+ The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
43
+ ```
44
+ hyperparameters = {
45
+ "help": false,
46
+ "algo": "APPO",
47
+ "env": "atari_asteroid",
48
+ "experiment": "atari_asteroid_APPO",
49
+ "train_dir": "./train_atari",
50
+ "restart_behavior": "restart",
51
+ "device": "gpu",
52
+ "seed": 1234,
53
+ "num_policies": 2,
54
+ "async_rl": true,
55
+ "serial_mode": false,
56
+ "batched_sampling": true,
57
+ "num_batches_to_accumulate": 2,
58
+ "worker_num_splits": 1,
59
+ "policy_workers_per_policy": 1,
60
+ "max_policy_lag": 1000,
61
+ "num_workers": 16,
62
+ "num_envs_per_worker": 2,
63
+ "batch_size": 1024,
64
+ "num_batches_per_epoch": 8,
65
+ "num_epochs": 4,
66
+ "rollout": 128,
67
+ "recurrence": 1,
68
+ "shuffle_minibatches": false,
69
+ "gamma": 0.99,
70
+ "reward_scale": 1.0,
71
+ "reward_clip": 1000.0,
72
+ "value_bootstrap": false,
73
+ "normalize_returns": true,
74
+ "exploration_loss_coeff": 0.0004677351413,
75
+ "value_loss_coeff": 0.5,
76
+ "kl_loss_coeff": 0.0,
77
+ "exploration_loss": "entropy",
78
+ "gae_lambda": 0.95,
79
+ "ppo_clip_ratio": 0.1,
80
+ "ppo_clip_value": 1.0,
81
+ "with_vtrace": true,
82
+ "vtrace_rho": 1.0,
83
+ "vtrace_c": 1.0,
84
+ "optimizer": "adam",
85
+ "adam_eps": 1e-05,
86
+ "adam_beta1": 0.9,
87
+ "adam_beta2": 0.999,
88
+ "max_grad_norm": 0.0,
89
+ "learning_rate": 0.0003033891184,
90
+ "lr_schedule": "linear_decay",
91
+ "lr_schedule_kl_threshold": 0.008,
92
+ "lr_adaptive_min": 1e-06,
93
+ "lr_adaptive_max": 0.01,
94
+ "obs_subtract_mean": 0.0,
95
+ "obs_scale": 255.0,
96
+ "normalize_input": true,
97
+ "normalize_input_keys": [
98
+ "obs"
99
+ ],
100
+ "decorrelate_experience_max_seconds": 0,
101
+ "decorrelate_envs_on_one_worker": true,
102
+ "actor_worker_gpus": [],
103
+ "set_workers_cpu_affinity": true,
104
+ "force_envs_single_thread": false,
105
+ "default_niceness": 0,
106
+ "log_to_file": true,
107
+ "experiment_summaries_interval": 3,
108
+ "flush_summaries_interval": 30,
109
+ "stats_avg": 100,
110
+ "summaries_use_frameskip": true,
111
+ "heartbeat_interval": 10,
112
+ "heartbeat_reporting_interval": 60,
113
+ "train_for_env_steps": 100000000,
114
+ "train_for_seconds": 10000000000,
115
+ "save_every_sec": 120,
116
+ "keep_checkpoints": 2,
117
+ "load_checkpoint_kind": "latest",
118
+ "save_milestones_sec": 1200,
119
+ "save_best_every_sec": 5,
120
+ "save_best_metric": "reward",
121
+ "save_best_after": 100000,
122
+ "benchmark": false,
123
+ "encoder_mlp_layers": [
124
+ 512,
125
+ 512
126
+ ],
127
+ "encoder_conv_architecture": "convnet_atari",
128
+ "encoder_conv_mlp_layers": [
129
+ 512
130
+ ],
131
+ "use_rnn": false,
132
+ "rnn_size": 512,
133
+ "rnn_type": "gru",
134
+ "rnn_num_layers": 1,
135
+ "decoder_mlp_layers": [],
136
+ "nonlinearity": "relu",
137
+ "policy_initialization": "orthogonal",
138
+ "policy_init_gain": 1.0,
139
+ "actor_critic_share_weights": true,
140
+ "adaptive_stddev": false,
141
+ "continuous_tanh_scale": 0.0,
142
+ "initial_stddev": 1.0,
143
+ "use_env_info_cache": false,
144
+ "env_gpu_actions": false,
145
+ "env_gpu_observations": true,
146
+ "env_frameskip": 4,
147
+ "env_framestack": 4,
148
+ "pixel_format": "CHW"
149
+ }
150
+
151
+ ```
152
+
153
+
154
+
155
+ A(n) **APPO** impala model trained on the **mujoco_humanoid** environment.
156
+
157
+ This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a
158
+ high throughput on-policy RL framework. I have been using
159
+ Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
160
+
161
+
162
+ ## Downloading the model
163
+
164
+ After installing Sample-Factory, download the model with:
165
+ ```
166
+ python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-mujoco_humanoid
167
+ ```
168
+
169
+
170
+ ## Using the model
171
+
172
+ To run the model after download, use the `enjoy` script corresponding to this environment:
173
+ ```
174
+ python -m sf_examples.mujoco.enjoy_mujoco --algo=APPO --env=mujoco_humanoid --train_dir=./train_dir --experiment=APPO-mujoco_humanoid
175
+ ```
176
+
177
+
178
+ You can also upload models to the Hugging Face Hub using the same script with the `--push_to_hub` flag.
179
+ See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
180
+
181
+ ## Training with this model
182
+
183
+ To continue training with this model, use the `train` script corresponding to this environment:
184
+ ```
185
+ python -m sf_examples.mujoco.train_mujoco --algo=APPO --env=mujoco_humanoid --train_dir=./train_dir --experiment=APPO-mujoco_humanoid --restart_behavior=resume --train_for_env_steps=10000000000
186
+ ```
187
+
188
+ Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
189
+
checkpoint_p0/best_001912832_489684992_reward_11497.130.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d42a25f0189ef894e0185fb2e6654e6da85ec47685a7c75a9e736b48d68fcc33
3
+ size 373375
checkpoint_p0/checkpoint_001953056_499982336.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:826dd525448ea843f129436a1619cb5f5522d9ccbeb4a117a2c16bfcca223652
3
+ size 373639
checkpoint_p0/checkpoint_001953136_500006912.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b12ff6c8514b58bdbcad718f2f094d8b2cfbbb7587e5fe28e0adb241e59271c4
3
+ size 373639
checkpoint_p0/milestones/checkpoint_000027040_6922240.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d6ac2f5b4951559dfaf7f6295ecbdedaf65f39189c7267d7aeb54e54fad6308
3
+ size 374275
checkpoint_p0/milestones/checkpoint_000054464_13942784.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a949732015833b3dfd4583b34508f1d4882e4825f49f8f71db212d238a65c0b
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000081856_20955136.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8bbe18e5fede49ea730dc814ce0a57df4886950e71b4296bef6fbf0b533f5809
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000109536_28041216.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a71d7eb7a3b30ca4c178bb74065063b5b70ddf1e8b721665cca8d0e5e51bba6
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000136928_35053568.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8064f7a6331d5a8760855360d6b4d785de9972c3e5fd7bc9a7666c7e7a3cb5b6
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000164128_42016768.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a34c43434069c2aedd12c14953d6fe00a262194d468e5acbbe064326d2c9e87
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000191712_49078272.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ce0fed318f3987ffd94da88435bb90f15637a6e4370fb1087756e702a3fd930
3
+ size 374319
checkpoint_p0/milestones/checkpoint_000219680_56238080.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e161fdd44a8e0c81cc3390c3a3ae94e0e81c8b394ffb6840cd4b460e098ebb8a
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000247616_63389696.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f05901d9439cbc168182353374402644b7f529478fce3ae848cd2586acca8b55
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000275232_70459392.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff193340813fe6eb4e9d610602aa3521d76753152fd582d6ff9eea16cf9e853e
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000302272_77381632.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f68f870e0653cda0e8ba08551919397690ca6aebba6a0c535e76eeee54cdbb4
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000329152_84262912.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99768a5527d12ea0b4da6af964e8f6ea19cd2d2ae91a9dbbbe71fdcab77cbc1e
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000355968_91127808.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aaf2112dfbda9ec4a8502b7d544c6ab3fad16120192c06f4325a76df79daa0d7
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000382688_97968128.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad32e67b88db3ba4c9e34445479a10c1e9ed409d25d013581310a5e9945d4032
3
+ size 374383
checkpoint_p0/milestones/checkpoint_000409184_104751104.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:012e3ca86c55288a5eb9273c3a6b445bfb413bfe0d24fd2ba0e6cc989230b9e4
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000435392_111460352.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c81a27ec2e16ca89516c8a90561c79723d53a581028ac5f24311dedefa920181
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000461888_118243328.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a48185527ece0c2a716724f7edc29a34f54060404ebe2dea41fcb3b4ce8a062c
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000488160_124968960.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:76e7b1ed7a0759c383468632d54a9e34e376bfb327293656f8025e95a5fdeb52
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000514304_131661824.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cecfb0f370a07abc2f99506659ff44005da3975bf4b99218d033ad57557eaa61
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000540512_138371072.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3d98027385cc9d947436432db8e1a4fa85cb14f6c726fd2dfa37d64175e9a98
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000566592_145047552.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c091e676eef0961a4b9ec50b9ebc69c1d43c6d5540cb8b08f8112e9a695c4608
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000592864_151773184.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9820d71e61526b4816144e3c6b3ed48fb63bdc2d1286f10ef491f58a4e328e7
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000618912_158441472.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e55d397c66673ca28c2d420a48098fc85a59753487f7d622f1f546e741af277
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000644960_165109760.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e144f0c04385578c212710e3e4c78f57f723958fb60a3df4566fe290c080aab
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000670912_171753472.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b0a7f430bcb3a9abc9ff356ff4a11db98882e2a9f3733cbb787caa2244e271aa
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000696768_178372608.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b8798fe3c04295736436ba35f71f0866c425cdb2297b11da15d7fc86174e671b
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000722592_184983552.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:02648da074d994bd29157ce61d8d9f9466d6d10d3744451b1c5ba88c9084a6dc
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000748192_191537152.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:776563606bcb7109b73c3c78f4a4895e7bc9b365471d4ee21b57a5c419a31b66
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000773504_198017024.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:69267e215d74ba15250e55802ef63214e22171e1a722a2a4c40127fd9220d536
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000799008_204546048.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17988df98b8cb4d41b71f7ade65c39a85832f20c54281979c855a3ce15382c6c
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000824576_211091456.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12a70e46cea922e5204e1182593e460d7f408ab919a45f2a3e7a92c4a12d70bc
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000850208_217653248.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe6318c915bb0ea24a24fdbc15de5b87defc02894f58420887c320d4d819aa48
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000875936_224239616.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f21a8b8fc42cb8cc5570e9788a3ee9c74240e5d8e2030e8abefc09592ffaa9c
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000901856_230875136.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7520638c449a13f515670c115eda8a325c0fa5c3ae6833c69e63fddc7223c01
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000927584_237461504.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c927387cd396f526d932033b7cb18ad5e96f5219a0c67c0f7f11de7f2a61f58
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000953440_244080640.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:22c5670c41f80e158ce612f956b38fc4e3f22c532fef1f5e07e5d46d2d552e54
3
+ size 374427
checkpoint_p0/milestones/checkpoint_000979360_250716160.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36b751108e4f96060d5bd8ca56e9ed9c87287949571f96081524629478901d58
3
+ size 374427
checkpoint_p0/milestones/checkpoint_001005376_257376256.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42b3c258bac968fb512596629aa0a4ef7c1496cf4a3ee123f8d48c2011fba121
3
+ size 374427
checkpoint_p0/milestones/checkpoint_001031168_263979008.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9c45d55355ec94b8e30cac995ddf2575b6aa0df2a38c6c615e368ef42f7c3962
3
+ size 374427
checkpoint_p0/milestones/checkpoint_001056704_270516224.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b26289feae9842fa4f7009631d4a074b654ef3f70c2e6db20b214cbf43cb4024
3
+ size 374427
checkpoint_p0/milestones/checkpoint_001082176_277037056.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63ede7b2315ac73d3d5c94537a3036ae60de901f8350024d385fcdb8580a79c2
3
+ size 374427
checkpoint_p0/milestones/checkpoint_001107840_283607040.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6590fd49a0c74b841c6bd8de01d2ca9213430478c4bbf2406f5a973c7154ede2
3
+ size 374427
checkpoint_p0/milestones/checkpoint_001133280_290119680.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec1d4042bfcc413c2d872ebb4d3677485042e0aadc5249e5e146863e6c1ace6e
3
+ size 374427