zhong-al commited on
Commit
aa912e2
1 Parent(s): cf8bf9f

Upload model

Browse files
Files changed (3) hide show
  1. README.md +199 -0
  2. config.json +643 -0
  3. model.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,199 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ tags: []
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+ This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
config.json ADDED
@@ -0,0 +1,643 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "X3DModel"
4
+ ],
5
+ "cfg": {
6
+ "AUG": {
7
+ "AA_TYPE": "rand-m9-mstd0.5-inc1",
8
+ "COLOR_JITTER": 0.4,
9
+ "ENABLE": false,
10
+ "GEN_MASK_LOADER": false,
11
+ "INTERPOLATION": "bicubic",
12
+ "MASK_FRAMES": false,
13
+ "MASK_RATIO": 0.0,
14
+ "MASK_TUBE": false,
15
+ "MASK_WINDOW_SIZE": [
16
+ 8,
17
+ 7,
18
+ 7
19
+ ],
20
+ "MAX_MASK_PATCHES_PER_BLOCK": null,
21
+ "NUM_SAMPLE": 1,
22
+ "RE_COUNT": 1,
23
+ "RE_MODE": "pixel",
24
+ "RE_PROB": 0.25,
25
+ "RE_SPLIT": false
26
+ },
27
+ "AVA": {
28
+ "ANNOTATION_DIR": "/mnt/vol/gfsai-flash3-east/ai-group/users/haoqifan/ava/frame_list/",
29
+ "BGR": false,
30
+ "DETECTION_SCORE_THRESH": 0.9,
31
+ "EXCLUSION_FILE": "ava_val_excluded_timestamps_v2.2.csv",
32
+ "FRAME_DIR": "/mnt/fair-flash3-east/ava_trainval_frames.img/",
33
+ "FRAME_LIST_DIR": "/mnt/vol/gfsai-flash3-east/ai-group/users/haoqifan/ava/frame_list/",
34
+ "FULL_TEST_ON_VAL": false,
35
+ "GROUNDTRUTH_FILE": "ava_val_v2.2.csv",
36
+ "IMG_PROC_BACKEND": "cv2",
37
+ "LABEL_MAP_FILE": "ava_action_list_v2.2_for_activitynet_2019.pbtxt",
38
+ "TEST_FORCE_FLIP": false,
39
+ "TEST_LISTS": [
40
+ "val.csv"
41
+ ],
42
+ "TEST_PREDICT_BOX_LISTS": [
43
+ "ava_val_predicted_boxes.csv"
44
+ ],
45
+ "TRAIN_GT_BOX_LISTS": [
46
+ "ava_train_v2.2.csv"
47
+ ],
48
+ "TRAIN_LISTS": [
49
+ "train.csv"
50
+ ],
51
+ "TRAIN_PCA_JITTER_ONLY": true,
52
+ "TRAIN_PREDICT_BOX_LISTS": [],
53
+ "TRAIN_USE_COLOR_AUGMENTATION": false
54
+ },
55
+ "BENCHMARK": {
56
+ "LOG_PERIOD": 100,
57
+ "NUM_EPOCHS": 5,
58
+ "SHUFFLE": true
59
+ },
60
+ "BN": {
61
+ "GLOBAL_SYNC": false,
62
+ "NORM_TYPE": "sync_batchnorm",
63
+ "NUM_BATCHES_PRECISE": 200,
64
+ "NUM_SPLITS": 1,
65
+ "NUM_SYNC_DEVICES": 1,
66
+ "USE_PRECISE_STATS": true,
67
+ "WEIGHT_DECAY": 0.0
68
+ },
69
+ "CONTRASTIVE": {
70
+ "BN_MLP": false,
71
+ "BN_SYNC_MLP": false,
72
+ "DELTA_CLIPS_MAX": Infinity,
73
+ "DELTA_CLIPS_MIN": -Infinity,
74
+ "DIM": 128,
75
+ "INTERP_MEMORY": false,
76
+ "KNN_ON": true,
77
+ "LENGTH": 239975,
78
+ "LOCAL_SHUFFLE_BN": true,
79
+ "MEM_TYPE": "1d",
80
+ "MLP_DIM": 2048,
81
+ "MOCO_MULTI_VIEW_QUEUE": false,
82
+ "MOMENTUM": 0.5,
83
+ "MOMENTUM_ANNEALING": false,
84
+ "NUM_CLASSES_DOWNSTREAM": 400,
85
+ "NUM_MLP_LAYERS": 1,
86
+ "PREDICTOR_DEPTHS": [],
87
+ "QUEUE_LEN": 65536,
88
+ "SEQUENTIAL": false,
89
+ "SIMCLR_DIST_ON": true,
90
+ "SWAV_QEUE_LEN": 0,
91
+ "T": 0.07,
92
+ "TYPE": "mem"
93
+ },
94
+ "DATA": {
95
+ "COLOR_RND_GRAYSCALE": 0.0,
96
+ "DECODING_BACKEND": "torchvision",
97
+ "DECODING_SHORT_SIZE": 256,
98
+ "DUMMY_LOAD": false,
99
+ "ENSEMBLE_METHOD": "max",
100
+ "IN22K_TRAINVAL": false,
101
+ "IN22k_VAL_IN1K": "",
102
+ "INPUT_CHANNEL_NUM": [
103
+ 3
104
+ ],
105
+ "INV_UNIFORM_SAMPLE": true,
106
+ "IN_VAL_CROP_RATIO": 0.875,
107
+ "LOADER_CHUNK_OVERALL_SIZE": 0,
108
+ "LOADER_CHUNK_SIZE": 0,
109
+ "MEAN": [
110
+ 0.45,
111
+ 0.45,
112
+ 0.45
113
+ ],
114
+ "MULTI_LABEL": true,
115
+ "NUM_FRAMES": 16,
116
+ "PATH_LABEL_SEPARATOR": " ",
117
+ "PATH_PREFIX": "kabr/KABR/dataset/image",
118
+ "PATH_TO_DATA_DIR": "kabr/KABR/annotation",
119
+ "PATH_TO_PRELOAD_IMDB": "",
120
+ "RANDOM_FLIP": true,
121
+ "REVERSE_INPUT_CHANNEL": true,
122
+ "SAMPLING_RATE": 5,
123
+ "SKIP_ROWS": 0,
124
+ "SSL_BLUR_SIGMA_MAX": [
125
+ 0.0,
126
+ 2.0
127
+ ],
128
+ "SSL_BLUR_SIGMA_MIN": [
129
+ 0.0,
130
+ 0.1
131
+ ],
132
+ "SSL_COLOR_BRI_CON_SAT": [
133
+ 0.2,
134
+ 0.2,
135
+ 0.2
136
+ ],
137
+ "SSL_COLOR_HUE": 0.1,
138
+ "SSL_COLOR_JITTER": true,
139
+ "SSL_MOCOV2_AUG": false,
140
+ "STD": [
141
+ 0.225,
142
+ 0.225,
143
+ 0.225
144
+ ],
145
+ "TARGET_FPS": 30,
146
+ "TEST_CROP_SIZE": 300,
147
+ "TIME_DIFF_PROB": 0.0,
148
+ "TRAIN_CROP_NUM_SPATIAL": 1,
149
+ "TRAIN_CROP_NUM_TEMPORAL": 1,
150
+ "TRAIN_CROP_SIZE": 300,
151
+ "TRAIN_JITTER_ASPECT_RELATIVE": [],
152
+ "TRAIN_JITTER_FPS": 0.0,
153
+ "TRAIN_JITTER_MOTION_SHIFT": false,
154
+ "TRAIN_JITTER_SCALES": [
155
+ 300,
156
+ 400
157
+ ],
158
+ "TRAIN_JITTER_SCALES_RELATIVE": [],
159
+ "TRAIN_PCA_EIGVAL": [
160
+ 0.225,
161
+ 0.224,
162
+ 0.229
163
+ ],
164
+ "TRAIN_PCA_EIGVEC": [
165
+ [
166
+ -0.5675,
167
+ 0.7192,
168
+ 0.4009
169
+ ],
170
+ [
171
+ -0.5808,
172
+ -0.0045,
173
+ -0.814
174
+ ],
175
+ [
176
+ -0.5836,
177
+ -0.6948,
178
+ 0.4203
179
+ ]
180
+ ],
181
+ "USE_OFFSET_SAMPLING": false
182
+ },
183
+ "DATA_LOADER": {
184
+ "ENABLE_MULTI_THREAD_DECODE": false,
185
+ "NUM_WORKERS": 8,
186
+ "PIN_MEMORY": true
187
+ },
188
+ "DEMO": {
189
+ "BUFFER_SIZE": 0,
190
+ "CLIP_VIS_SIZE": 10,
191
+ "COMMON_CLASS_NAMES": [
192
+ "watch (a person)",
193
+ "talk to (e.g., self, a person, a group)",
194
+ "listen to (a person)",
195
+ "touch (an object)",
196
+ "carry/hold (an object)",
197
+ "walk",
198
+ "sit",
199
+ "lie/sleep",
200
+ "bend/bow (at the waist)"
201
+ ],
202
+ "COMMON_CLASS_THRES": 0.7,
203
+ "DETECTRON2_CFG": "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml",
204
+ "DETECTRON2_THRESH": 0.9,
205
+ "DETECTRON2_WEIGHTS": "detectron2://COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl",
206
+ "DISPLAY_HEIGHT": 0,
207
+ "DISPLAY_WIDTH": 0,
208
+ "ENABLE": false,
209
+ "FPS": 30,
210
+ "GT_BOXES": "",
211
+ "INPUT_FORMAT": "BGR",
212
+ "INPUT_VIDEO": "kabr/KABR/dataset/video/G0103.mp4",
213
+ "LABEL_FILE_PATH": "kabr/KABR/annotation/classes.json",
214
+ "NUM_CLIPS_SKIP": 1,
215
+ "NUM_VIS_INSTANCES": 1,
216
+ "OUTPUT_FILE": "kabr/KABR/dataset/predict/G0103.mp4",
217
+ "OUTPUT_FPS": -1,
218
+ "PREDS_BOXES": "",
219
+ "SLOWMO": 1,
220
+ "STARTING_SECOND": 900,
221
+ "THREAD_ENABLE": false,
222
+ "UNCOMMON_CLASS_THRES": 0.3,
223
+ "VIS_MODE": "thres",
224
+ "WEBCAM": -1
225
+ },
226
+ "DETECTION": {
227
+ "ALIGNED": true,
228
+ "ENABLE": false,
229
+ "ROI_XFORM_RESOLUTION": 7,
230
+ "SPATIAL_SCALE_FACTOR": 16
231
+ },
232
+ "DIST_BACKEND": "nccl",
233
+ "LOG_MODEL_INFO": true,
234
+ "LOG_PERIOD": 10,
235
+ "MASK": {
236
+ "DECODER_DEPTH": 0,
237
+ "DECODER_EMBED_DIM": 512,
238
+ "DECODER_SEP_POS_EMBED": false,
239
+ "DEC_KV_KERNEL": [],
240
+ "DEC_KV_STRIDE": [],
241
+ "ENABLE": false,
242
+ "HEAD_TYPE": "separate",
243
+ "MAE_ON": false,
244
+ "MAE_RND_MASK": false,
245
+ "NORM_PRED_PIXEL": true,
246
+ "PER_FRAME_MASKING": false,
247
+ "PRED_HOG": false,
248
+ "PRETRAIN_DEPTH": [
249
+ 15
250
+ ],
251
+ "SCALE_INIT_BY_DEPTH": false,
252
+ "TIME_STRIDE_LOSS": true
253
+ },
254
+ "MIXUP": {
255
+ "ALPHA": 0.8,
256
+ "CUTMIX_ALPHA": 1.0,
257
+ "ENABLE": false,
258
+ "LABEL_SMOOTH_VALUE": 0.1,
259
+ "PROB": 1.0,
260
+ "SWITCH_PROB": 0.5
261
+ },
262
+ "MODEL": {
263
+ "ACT_CHECKPOINT": false,
264
+ "ARCH": "x3d",
265
+ "DETACH_FINAL_FC": false,
266
+ "DROPCONNECT_RATE": 0.0,
267
+ "DROPOUT_RATE": 0.5,
268
+ "FC_INIT_STD": 0.01,
269
+ "FP16_ALLREDUCE": false,
270
+ "FROZEN_BN": false,
271
+ "HEAD_ACT": "sigmoid",
272
+ "LOSS_FUNC": "EQL",
273
+ "MODEL_NAME": "X3D",
274
+ "MULTI_PATHWAY_ARCH": [
275
+ "slowfast"
276
+ ],
277
+ "NUM_CLASSES": 8,
278
+ "SINGLE_PATHWAY_ARCH": [
279
+ "2d",
280
+ "c2d",
281
+ "i3d",
282
+ "slow",
283
+ "x3d",
284
+ "mvit",
285
+ "maskmvit"
286
+ ]
287
+ },
288
+ "MULTIGRID": {
289
+ "BN_BASE_SIZE": 8,
290
+ "DEFAULT_B": 0,
291
+ "DEFAULT_S": 0,
292
+ "DEFAULT_T": 0,
293
+ "EPOCH_FACTOR": 1.5,
294
+ "EVAL_FREQ": 3,
295
+ "LONG_CYCLE": false,
296
+ "LONG_CYCLE_FACTORS": [
297
+ [
298
+ 0.25,
299
+ 0.7071067811865476
300
+ ],
301
+ [
302
+ 0.5,
303
+ 0.7071067811865476
304
+ ],
305
+ [
306
+ 0.5,
307
+ 1
308
+ ],
309
+ [
310
+ 1,
311
+ 1
312
+ ]
313
+ ],
314
+ "LONG_CYCLE_SAMPLING_RATE": 0,
315
+ "SHORT_CYCLE": false,
316
+ "SHORT_CYCLE_FACTORS": [
317
+ 0.5,
318
+ 0.7071067811865476
319
+ ]
320
+ },
321
+ "MVIT": {
322
+ "CLS_EMBED_ON": true,
323
+ "DEPTH": 16,
324
+ "DIM_MUL": [],
325
+ "DIM_MUL_IN_ATT": false,
326
+ "DROPOUT_RATE": 0.0,
327
+ "DROPPATH_RATE": 0.1,
328
+ "EMBED_DIM": 96,
329
+ "HEAD_INIT_SCALE": 1.0,
330
+ "HEAD_MUL": [],
331
+ "LAYER_SCALE_INIT_VALUE": 0.0,
332
+ "MLP_RATIO": 4.0,
333
+ "MODE": "conv",
334
+ "NORM": "layernorm",
335
+ "NORM_STEM": false,
336
+ "NUM_HEADS": 1,
337
+ "PATCH_2D": false,
338
+ "PATCH_KERNEL": [
339
+ 3,
340
+ 7,
341
+ 7
342
+ ],
343
+ "PATCH_PADDING": [
344
+ 2,
345
+ 4,
346
+ 4
347
+ ],
348
+ "PATCH_STRIDE": [
349
+ 2,
350
+ 4,
351
+ 4
352
+ ],
353
+ "POOL_FIRST": false,
354
+ "POOL_KVQ_KERNEL": null,
355
+ "POOL_KV_STRIDE": [],
356
+ "POOL_KV_STRIDE_ADAPTIVE": null,
357
+ "POOL_Q_STRIDE": [],
358
+ "QKV_BIAS": true,
359
+ "REL_POS_SPATIAL": false,
360
+ "REL_POS_TEMPORAL": false,
361
+ "REL_POS_ZERO_INIT": false,
362
+ "RESIDUAL_POOLING": false,
363
+ "REV": {
364
+ "BUFFER_LAYERS": [],
365
+ "ENABLE": false,
366
+ "PRE_Q_FUSION": "avg",
367
+ "RESPATH_FUSE": "concat",
368
+ "RES_PATH": "conv"
369
+ },
370
+ "SEPARATE_QKV": false,
371
+ "SEP_POS_EMBED": false,
372
+ "USE_ABS_POS": true,
373
+ "USE_FIXED_SINCOS_POS": false,
374
+ "USE_MEAN_POOLING": false,
375
+ "ZERO_DECAY_POS_CLS": true
376
+ },
377
+ "NONLOCAL": {
378
+ "GROUP": [
379
+ [
380
+ 1
381
+ ],
382
+ [
383
+ 1
384
+ ],
385
+ [
386
+ 1
387
+ ],
388
+ [
389
+ 1
390
+ ]
391
+ ],
392
+ "INSTANTIATION": "dot_product",
393
+ "LOCATION": [
394
+ [
395
+ []
396
+ ],
397
+ [
398
+ []
399
+ ],
400
+ [
401
+ []
402
+ ],
403
+ [
404
+ []
405
+ ]
406
+ ],
407
+ "POOL": [
408
+ [
409
+ [
410
+ 1,
411
+ 2,
412
+ 2
413
+ ],
414
+ [
415
+ 1,
416
+ 2,
417
+ 2
418
+ ]
419
+ ],
420
+ [
421
+ [
422
+ 1,
423
+ 2,
424
+ 2
425
+ ],
426
+ [
427
+ 1,
428
+ 2,
429
+ 2
430
+ ]
431
+ ],
432
+ [
433
+ [
434
+ 1,
435
+ 2,
436
+ 2
437
+ ],
438
+ [
439
+ 1,
440
+ 2,
441
+ 2
442
+ ]
443
+ ],
444
+ [
445
+ [
446
+ 1,
447
+ 2,
448
+ 2
449
+ ],
450
+ [
451
+ 1,
452
+ 2,
453
+ 2
454
+ ]
455
+ ]
456
+ ]
457
+ },
458
+ "NUM_GPUS": 0,
459
+ "NUM_SHARDS": 1,
460
+ "OUTPUT_DIR": "kabr/KABR/logs/x3d-l-kabr",
461
+ "RESNET": {
462
+ "DEPTH": 50,
463
+ "INPLACE_RELU": true,
464
+ "NUM_BLOCK_TEMP_KERNEL": [
465
+ [
466
+ 3
467
+ ],
468
+ [
469
+ 4
470
+ ],
471
+ [
472
+ 6
473
+ ],
474
+ [
475
+ 3
476
+ ]
477
+ ],
478
+ "NUM_GROUPS": 1,
479
+ "SPATIAL_DILATIONS": [
480
+ [
481
+ 1
482
+ ],
483
+ [
484
+ 1
485
+ ],
486
+ [
487
+ 1
488
+ ],
489
+ [
490
+ 1
491
+ ]
492
+ ],
493
+ "SPATIAL_STRIDES": [
494
+ [
495
+ 1
496
+ ],
497
+ [
498
+ 2
499
+ ],
500
+ [
501
+ 2
502
+ ],
503
+ [
504
+ 2
505
+ ]
506
+ ],
507
+ "STRIDE_1X1": false,
508
+ "TRANS_FUNC": "x3d_transform",
509
+ "WIDTH_PER_GROUP": 64,
510
+ "ZERO_INIT_FINAL_BN": true,
511
+ "ZERO_INIT_FINAL_CONV": false
512
+ },
513
+ "RNG_SEED": 0,
514
+ "SHARD_ID": 0,
515
+ "SLOWFAST": {
516
+ "ALPHA": 8,
517
+ "BETA_INV": 8,
518
+ "FUSION_CONV_CHANNEL_RATIO": 2,
519
+ "FUSION_KERNEL_SZ": 5
520
+ },
521
+ "SOLVER": {
522
+ "BASE_LR": 0.05,
523
+ "BASE_LR_SCALE_NUM_SHARDS": true,
524
+ "BETAS": [
525
+ 0.9,
526
+ 0.999
527
+ ],
528
+ "CLIP_GRAD_L2NORM": null,
529
+ "CLIP_GRAD_VAL": null,
530
+ "COSINE_AFTER_WARMUP": false,
531
+ "COSINE_END_LR": 0.0,
532
+ "DAMPENING": 0.0,
533
+ "GAMMA": 0.1,
534
+ "LARS_ON": false,
535
+ "LAYER_DECAY": 1.0,
536
+ "LRS": [],
537
+ "LR_POLICY": "cosine",
538
+ "MAX_EPOCH": 120,
539
+ "MOMENTUM": 0.9,
540
+ "NESTEROV": true,
541
+ "OPTIMIZING_METHOD": "sgd",
542
+ "STEPS": [],
543
+ "STEP_SIZE": 1,
544
+ "WARMUP_EPOCHS": 35.0,
545
+ "WARMUP_FACTOR": 0.1,
546
+ "WARMUP_START_LR": 0.01,
547
+ "WEIGHT_DECAY": 5e-05,
548
+ "ZERO_WD_1D_PARAM": false
549
+ },
550
+ "TASK": "",
551
+ "TENSORBOARD": {
552
+ "CATEGORIES_PATH": "",
553
+ "CLASS_NAMES_PATH": "kabr/KABR/annotation/classes.json",
554
+ "CONFUSION_MATRIX": {
555
+ "ENABLE": true,
556
+ "FIGSIZE": [
557
+ 8,
558
+ 8
559
+ ],
560
+ "SUBSET_PATH": "kabr/KABR/annotation/classes.txt"
561
+ },
562
+ "ENABLE": true,
563
+ "HISTOGRAM": {
564
+ "ENABLE": true,
565
+ "FIGSIZE": [
566
+ 8,
567
+ 8
568
+ ],
569
+ "SUBSET_PATH": "kabr/KABR/annotation/classes.txt",
570
+ "TOPK": 3
571
+ },
572
+ "LOG_DIR": "",
573
+ "MODEL_VIS": {
574
+ "ACTIVATIONS": true,
575
+ "COLORMAP": "Pastel2",
576
+ "ENABLE": true,
577
+ "GRAD_CAM": {
578
+ "COLORMAP": "viridis",
579
+ "ENABLE": true,
580
+ "LAYER_LIST": [
581
+ "s5/pathway0_res14"
582
+ ],
583
+ "USE_TRUE_LABEL": false
584
+ },
585
+ "INPUT_VIDEO": true,
586
+ "LAYER_LIST": [
587
+ "s5/pathway0_res14"
588
+ ],
589
+ "MODEL_WEIGHTS": true,
590
+ "TOPK_PREDS": 1
591
+ },
592
+ "PREDICTIONS_PATH": "",
593
+ "WRONG_PRED_VIS": {
594
+ "ENABLE": false,
595
+ "SUBSET_PATH": "",
596
+ "TAG": "Incorrectly classified videos."
597
+ }
598
+ },
599
+ "TEST": {
600
+ "BATCH_SIZE": 64,
601
+ "CHECKPOINT_FILE_PATH": "",
602
+ "CHECKPOINT_TYPE": "pytorch",
603
+ "DATASET": "charades",
604
+ "ENABLE": false,
605
+ "NUM_ENSEMBLE_VIEWS": 2,
606
+ "NUM_SPATIAL_CROPS": 1,
607
+ "NUM_TEMPORAL_CLIPS": [],
608
+ "SAVE_RESULTS_PATH": "kabr/KABR/logs/x3d-l-kabr/results.txt"
609
+ },
610
+ "TRAIN": {
611
+ "AUTO_RESUME": true,
612
+ "BATCH_SIZE": 64,
613
+ "CHECKPOINT_CLEAR_NAME_PATTERN": [],
614
+ "CHECKPOINT_EPOCH_RESET": true,
615
+ "CHECKPOINT_FILE_PATH": "slowfast/projects/x3d/x3d_l.pyth",
616
+ "CHECKPOINT_INFLATE": false,
617
+ "CHECKPOINT_IN_INIT": false,
618
+ "CHECKPOINT_PERIOD": 5,
619
+ "CHECKPOINT_TYPE": "pytorch",
620
+ "DATASET": "charades",
621
+ "ENABLE": true,
622
+ "EVAL_PERIOD": 5,
623
+ "KILL_LOSS_EXPLOSION_FACTOR": 0.0,
624
+ "MIXED_PRECISION": false
625
+ },
626
+ "VIS_MASK": {
627
+ "ENABLE": false
628
+ },
629
+ "X3D": {
630
+ "BN_LIN5": false,
631
+ "BOTTLENECK_FACTOR": 2.25,
632
+ "CHANNELWISE_3x3x3": true,
633
+ "DEPTH_FACTOR": 5.0,
634
+ "DIM_C1": 12,
635
+ "DIM_C5": 2048,
636
+ "SCALE_RES2": false,
637
+ "WIDTH_FACTOR": 2.0
638
+ }
639
+ },
640
+ "model_type": "x3d",
641
+ "torch_dtype": "float32",
642
+ "transformers_version": "4.46.0"
643
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3b84df54f08ad0b9a7858524fdb144229dddcd59b2a82c008b7832a9c6755a9
3
+ size 21795296