add

.gitattributes

@@ -0,0 +1,27 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text 
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,113 @@
+---
+language:
+- en
+datasets:
+- c4
+tags:
+- deep-narrow
+inference: false
+
+license: apache-2.0
+---
+
+# T5-Efficient-TINY (Deep-Narrow version)
+
+T5-Efficient-TINY is a variation of [Google's original T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) following the [T5 model architecture](https://huggingface.co/docs/transformers/model_doc/t5).
+It is a *pretrained-only* checkpoint and was released with the
+paper **[Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers](https://arxiv.org/abs/2109.10686)**
+by *Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler*.
+
+In a nutshell, the paper indicates that a **Deep-Narrow** model architecture is favorable for **downstream** performance compared to other model architectures
+of similar parameter count.
+
+To quote the paper:
+
+> We generally recommend a DeepNarrow strategy where the model’s depth is preferentially increased
+> before considering any other forms of uniform scaling across other dimensions. This is largely due to
+> how much depth influences the Pareto-frontier as shown in earlier sections of the paper. Specifically, a
+> tall small (deep and narrow) model is generally more efficient compared to the base model. Likewise,
+> a tall base model might also generally more efficient compared to a large model. We generally find
+> that, regardless of size, even if absolute performance might increase as we continue to stack layers,
+> the relative gain of Pareto-efficiency diminishes as we increase the layers, converging at 32 to 36
+> layers. Finally, we note that our notion of efficiency here relates to any one compute dimension, i.e.,
+> params, FLOPs or throughput (speed). We report all three key efficiency metrics (number of params,
+> FLOPS and speed) and leave this decision to the practitioner to decide which compute dimension to
+> consider.
+
+To be more precise, *model depth* is defined as the number of transformer blocks that are stacked sequentially.
+A sequence of word embeddings is therefore processed sequentially by each transformer block.
+
+## Details model architecture
+
+This model checkpoint - **t5-efficient-tiny** - is of model type **Tiny** with no variations.
+It has **15.58** million parameters and thus requires *ca.* **62.32 MB** of memory in full precision (*fp32*)
+ or  **31.16 MB** of memory in half precision (*fp16* or *bf16*).
+
+A summary of the *original* T5 model architectures can be seen here:
+
+| Model | nl (el/dl) | ff | dm | kv | nh | #Params|
+| ----| ---- | ---- | ---- | ---- | ---- | ----|
+| Tiny | 4/4 | 1024 | 256 | 32 | 4 | 16M|
+| Mini | 4/4 | 1536 | 384 | 32 | 8 | 31M|
+| Small | 6/6 | 2048 | 512 | 32 | 8 | 60M|
+| Base | 12/12 | 3072 | 768 | 64 | 12 | 220M|
+| Large | 24/24 | 4096 | 1024 | 64 | 16 | 738M|
+| Xl | 24/24 | 16384 | 1024 | 128 | 32 | 3B|
+| XXl | 24/24 | 65536 | 1024 | 128 | 128 | 11B|
+
+whereas the following abbreviations are used:
+
+| Abbreviation | Definition |
+| ----| ---- |
+| nl | Number of transformer blocks (depth) |
+| dm | Dimension of embedding vector (output vector of transformers block) |
+| kv | Dimension of key/value projection matrix |
+| nh | Number of attention heads |
+| ff | Dimension of intermediate vector within transformer block (size of feed-forward projection matrix) | 
+| el | Number of transformer blocks in the encoder (encoder depth) | 
+| dl | Number of transformer blocks in the decoder (decoder depth) | 
+| sh | Signifies that attention heads are shared | 
+| skv | Signifies that key-values projection matrices are tied | 
+
+If a model checkpoint has no specific, *el* or *dl* than both the number of encoder- and decoder layers correspond to *nl*.
+
+## Pre-Training
+
+The checkpoint was pretrained on the [Colossal, Cleaned version of Common Crawl (C4)](https://huggingface.co/datasets/c4) for 524288 steps using 
+the span-based masked language modeling (MLM) objective.
+
+## Fine-Tuning
+
+**Note**: This model is a **pretrained** checkpoint and has to be fine-tuned for practical usage.
+The checkpoint was pretrained in English and is therefore only useful for English NLP tasks.
+You can follow on of the following examples on how to fine-tune the model:
+
+*PyTorch*:
+
+- [Summarization](https://github.com/huggingface/transformers/tree/master/examples/pytorch/summarization)
+- [Question Answering](https://github.com/huggingface/transformers/blob/master/examples/pytorch/question-answering/run_seq2seq_qa.py)
+- [Text Classification](https://github.com/huggingface/transformers/tree/master/examples/pytorch/text-classification) - *Note*: You will have to slightly adapt the training example here to make it work with an encoder-decoder model.
+
+*Tensorflow*:
+
+- [Summarization](https://github.com/huggingface/transformers/tree/master/examples/tensorflow/summarization)
+- [Text Classification](https://github.com/huggingface/transformers/tree/master/examples/tensorflow/text-classification) - *Note*: You will have to slightly adapt the training example here to make it work with an encoder-decoder model.
+
+*JAX/Flax*:
+
+- [Summarization](https://github.com/huggingface/transformers/tree/master/examples/flax/summarization)
+- [Text Classification](https://github.com/huggingface/transformers/tree/master/examples/flax/text-classification) - *Note*: You will have to slightly adapt the training example here to make it work with an encoder-decoder model.
+
+## Downstream Performance
+
+TODO: Add table if available
+
+## Computational Complexity
+
+TODO: Add table if available
+
+## More information
+
+We strongly recommend the reader to go carefully through the original paper **[Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers](https://arxiv.org/abs/2109.10686)** to get a more nuanced understanding of this model checkpoint.
+As explained in the following [issue](https://github.com/google-research/google-research/issues/986#issuecomment-1035051145), checkpoints including the *sh* or *skv* 
+model architecture variations have *not* been ported to Transformers as they are probably of limited practical usage and are lacking a more detailed description. Those checkpoints are kept [here](https://huggingface.co/NewT5SharedHeadsSharedKeyValues) as they might be ported potentially in the future.

config.json

@@ -0,0 +1,27 @@
+{
+  "_name_or_path": "t5-efficient-tiny",
+  "architectures": [
+    "T5ForConditionalGeneration"
+  ],
+  "d_ff": 1024,
+  "d_kv": 64,
+  "d_model": 256,
+  "decoder_start_token_id": 0,
+  "dropout_rate": 0.1,
+  "eos_token_id": 1,
+  "feed_forward_proj": "relu",
+  "initializer_factor": 1.0,
+  "is_encoder_decoder": true,
+  "layer_norm_epsilon": 1e-06,
+  "model_type": "t5",
+  "n_positions": 512,
+  "num_decoder_layers": 4,
+  "num_heads": 4,
+  "num_layers": 4,
+  "pad_token_id": 0,
+  "relative_attention_num_buckets": 32,
+  "torch_dtype": "float32",
+  "transformers_version": "4.17.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32128
+}

flax_model.msgpack

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16e959d18596c0cdd9da07fd4fb913f5f9aa7835decfd6a4b9f9fd96e960da26
+size 62286648

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "decoder_start_token_id": 0,
+  "eos_token_id": 1,
+  "pad_token_id": 0,
+  "transformers_version": "4.27.0.dev0"
+}

operative_config.gin

@@ -0,0 +1,370 @@
+import mesh_tensorflow.optimize
+import mesh_tensorflow.transformer.dataset
+import mesh_tensorflow.transformer.learning_rate_schedules
+import mesh_tensorflow.transformer.t2t_vocabulary
+import mesh_tensorflow.transformer.transformer
+import mesh_tensorflow.transformer.transformer_layers
+import mesh_tensorflow.transformer.utils
+import t5.models.mesh_transformer
+
+# Macros:
+# ==============================================================================
+d_ff = 1024
+d_kv = 64
+d_model = 256
+dropout_rate = 0.0
+inputs_length = 512
+mean_noise_span_length = 3.0
+MIXTURE_NAME = 'c4_v220_unsupervised'
+noise_density = 0.15
+num_heads = 4
+num_layers = 4
+
+# Parameters for adafactor_decay_rate_pow:
+# ==============================================================================
+adafactor_decay_rate_pow.offset = 0
+
+# Parameters for AdafactorOptimizer:
+# ==============================================================================
+AdafactorOptimizer.beta1 = 0.0
+AdafactorOptimizer.clipping_threshold = 1.0
+AdafactorOptimizer.decay_rate = None
+AdafactorOptimizer.epsilon1 = 1e-30
+AdafactorOptimizer.epsilon2 = 0.001
+AdafactorOptimizer.factored = True
+AdafactorOptimizer.min_dim_size_to_factor = 128
+AdafactorOptimizer.multiply_by_parameter_scale = True
+
+# Parameters for Bitransformer:
+# ==============================================================================
+Bitransformer.shared_embedding = True
+
+# Parameters for denoise:
+# ==============================================================================
+denoise.inputs_fn = @preprocessors.noise_span_to_unique_sentinel
+denoise.noise_density = %noise_density
+denoise.noise_mask_fn = @preprocessors.random_spans_noise_mask
+denoise.targets_fn = @preprocessors.nonnoise_span_to_unique_sentinel
+
+# Parameters for decoder/DenseReluDense:
+# ==============================================================================
+decoder/DenseReluDense.activation = 'relu'
+decoder/DenseReluDense.dropout_rate = %dropout_rate
+decoder/DenseReluDense.hidden_size = %d_ff
+decoder/DenseReluDense.use_bias = False
+
+# Parameters for encoder/DenseReluDense:
+# ==============================================================================
+encoder/DenseReluDense.activation = 'relu'
+encoder/DenseReluDense.dropout_rate = %dropout_rate
+encoder/DenseReluDense.hidden_size = %d_ff
+encoder/DenseReluDense.use_bias = False
+
+# Parameters for enc_dec_attention:
+# ==============================================================================
+# None.
+
+# Parameters for enc_dec_attention_bias:
+# ==============================================================================
+# None.
+
+# Parameters for decoder/EncDecAttention:
+# ==============================================================================
+decoder/EncDecAttention.relative_attention_type = None
+
+# Parameters for get_variable_dtype:
+# ==============================================================================
+get_variable_dtype.activation_dtype = 'bfloat16'
+
+# Parameters for get_vocab_embedding_cls:
+# ==============================================================================
+# None.
+
+# Parameters for get_vocabulary:
+# ==============================================================================
+get_vocabulary.mixture_or_task_name = %MIXTURE_NAME
+
+# Parameters for decoder/LayerStack:
+# ==============================================================================
+decoder/LayerStack.dropout_rate = None
+decoder/LayerStack.norm_epsilon = None
+decoder/LayerStack.recompute_grads = False
+decoder/LayerStack.sublayers_final = \
+[@transformer.sublayer_rms_norm, @transformer.sublayer_dropout]
+decoder/LayerStack.sublayers_initial = [@transformer.sublayer_dropout]
+decoder/LayerStack.sublayers_per_layer = \
+[@transformer.sublayer_rms_norm,
+@transformer.sublayer_call_layer,
+@transformer.sublayer_dropout,
+@transformer.sublayer_residual]
+
+# Parameters for encoder/LayerStack:
+# ==============================================================================
+encoder/LayerStack.dropout_rate = None
+encoder/LayerStack.norm_epsilon = None
+encoder/LayerStack.recompute_grads = False
+encoder/LayerStack.sublayers_final = \
+[@transformer.sublayer_rms_norm, @transformer.sublayer_dropout]
+encoder/LayerStack.sublayers_initial = [@transformer.sublayer_dropout]
+encoder/LayerStack.sublayers_per_layer = \
+[@transformer.sublayer_rms_norm,
+@transformer.sublayer_call_layer,
+@transformer.sublayer_dropout,
+@transformer.sublayer_residual]
+
+# Parameters for learning_rate_schedule_noam:
+# ==============================================================================
+learning_rate_schedule_noam.linear_decay_fraction = 0.0
+learning_rate_schedule_noam.multiplier = 1.0
+learning_rate_schedule_noam.offset = 0
+learning_rate_schedule_noam.warmup_steps = 10000
+
+# Parameters for make_bitransformer:
+# ==============================================================================
+make_bitransformer.decoder_name = 'decoder'
+make_bitransformer.encoder_name = 'encoder'
+
+# Parameters for decoder/make_layer_stack:
+# ==============================================================================
+decoder/make_layer_stack.block_scope = True
+decoder/make_layer_stack.layers = \
+[@mesh_tensorflow.transformer.transformer_layers.SelfAttention,
+@mesh_tensorflow.transformer.transformer_layers.EncDecAttention,
+@mesh_tensorflow.transformer.transformer_layers.DenseReluDense]
+decoder/make_layer_stack.num_layers = %num_layers
+
+# Parameters for encoder/make_layer_stack:
+# ==============================================================================
+encoder/make_layer_stack.block_scope = True
+encoder/make_layer_stack.layers = \
+[@mesh_tensorflow.transformer.transformer_layers.SelfAttention,
+@mesh_tensorflow.transformer.transformer_layers.DenseReluDense]
+encoder/make_layer_stack.num_layers = %num_layers
+
+# Parameters for mesh_train_dataset_fn:
+# ==============================================================================
+mesh_train_dataset_fn.mixture_or_task_name = %MIXTURE_NAME
+mesh_train_dataset_fn.pack = True
+mesh_train_dataset_fn.seed = None
+mesh_train_dataset_fn.use_cached = 1
+
+# Parameters for noise_span_to_unique_sentinel:
+# ==============================================================================
+# None.
+
+# Parameters for nonnoise_span_to_unique_sentinel:
+# ==============================================================================
+# None.
+
+# Parameters for pack_dataset:
+# ==============================================================================
+pack_dataset.use_custom_ops = True
+
+# Parameters for pack_or_pad:
+# ==============================================================================
+# None.
+
+# Parameters for random_spans_helper:
+# ==============================================================================
+random_spans_helper.extra_tokens_per_span_inputs = 1
+random_spans_helper.extra_tokens_per_span_targets = 1
+random_spans_helper.inputs_length = %inputs_length
+random_spans_helper.mean_noise_span_length = %mean_noise_span_length
+random_spans_helper.noise_density = %noise_density
+random_spans_helper.verbose = False
+
+# Parameters for random_spans_noise_mask:
+# ==============================================================================
+random_spans_noise_mask.mean_noise_span_length = %mean_noise_span_length
+
+# Parameters for random_spans_tokens_length:
+# ==============================================================================
+# None.
+
+# Parameters for reduce_concat_tokens:
+# ==============================================================================
+reduce_concat_tokens.batch_size = 128
+reduce_concat_tokens.feature_key = 'targets'
+
+# Parameters for rewrite_stack_variables:
+# ==============================================================================
+rewrite_stack_variables.max_combined_variable_size = 536870912
+
+# Parameters for run:
+# ==============================================================================
+run.autostack = True
+run.batch_size = ('tokens_per_batch', 65536)
+run.dataset_split = 'train'
+run.ensemble_inputs = None
+run.eval_checkpoint_step = None
+run.eval_dataset_fn = None
+run.eval_summary_dir = None
+run.export_checkpoint_step = None
+run.export_path = ''
+run.init_checkpoint = None
+run.iterations_per_loop = 100
+run.keep_checkpoint_max = None
+run.layout_rules = \
+'ensemble:ensemble,batch:batch,d_ff:model,heads:model,vocab:model,experts:batch'
+run.learning_rate_schedule = @learning_rate_schedules.learning_rate_schedule_noam
+run.mesh_devices = None
+run.mesh_shape = @mesh_tensorflow.transformer.utils.tpu_mesh_shape()
+run.mode = 'train'
+run.model_type = 'bitransformer'
+run.optimizer = @optimize.AdafactorOptimizer
+run.output_eval_examples = True
+run.perplexity_eval_steps = 100
+run.predict_fn = None
+run.save_checkpoints_steps = 5000
+run.seen_data_init_step = 0
+run.sequence_length = {'inputs': 512, 'targets': 128}
+run.skip_seen_data = False
+run.total_run_steps = None
+run.train_dataset_fn = @t5.models.mesh_transformer.mesh_train_dataset_fn
+run.train_steps = 524288
+run.variable_filter = None
+
+# Parameters for select_random_chunk:
+# ==============================================================================
+select_random_chunk.additional_feature_keys = None
+select_random_chunk.additional_passthrough_keys = None
+select_random_chunk.feature_key = 'targets'
+select_random_chunk.max_length = 65536
+select_random_chunk.uniform_random_start = False
+
+# Parameters for decoder/SelfAttention:
+# ==============================================================================
+decoder/SelfAttention.attention_func = None
+decoder/SelfAttention.attention_kwargs = None
+decoder/SelfAttention.combine_dims = True
+decoder/SelfAttention.dropout_rate = %dropout_rate
+decoder/SelfAttention.fold_scaling_into_initializer = True
+decoder/SelfAttention.keep_query_heads_dims = False
+decoder/SelfAttention.key_value_size = %d_kv
+decoder/SelfAttention.num_heads = %num_heads
+decoder/SelfAttention.num_memory_heads = 0
+decoder/SelfAttention.relative_attention_num_buckets = 32
+decoder/SelfAttention.relative_attention_type = 'bias_shared'
+decoder/SelfAttention.shared_kv = False
+
+# Parameters for encoder/SelfAttention:
+# ==============================================================================
+encoder/SelfAttention.attention_func = None
+encoder/SelfAttention.attention_kwargs = None
+encoder/SelfAttention.combine_dims = True
+encoder/SelfAttention.dropout_rate = %dropout_rate
+encoder/SelfAttention.fold_scaling_into_initializer = True
+encoder/SelfAttention.keep_query_heads_dims = False
+encoder/SelfAttention.key_value_size = %d_kv
+encoder/SelfAttention.num_heads = %num_heads
+encoder/SelfAttention.num_memory_heads = 0
+encoder/SelfAttention.relative_attention_num_buckets = 32
+encoder/SelfAttention.relative_attention_type = 'bias_shared'
+encoder/SelfAttention.shared_kv = False
+
+# Parameters for serialize_num_microbatches:
+# ==============================================================================
+serialize_num_microbatches.tokens_per_microbatch_per_replica = 8192
+
+# Parameters for SimdMeshImpl:
+# ==============================================================================
+SimdMeshImpl.allreduce_in_bfloat16_max_group_size = 8
+
+# Parameters for split_tokens:
+# ==============================================================================
+split_tokens.additional_feature_keys = None
+split_tokens.feature_key = 'targets'
+split_tokens.max_tokens_per_segment = @preprocessors.random_spans_tokens_length()
+split_tokens.min_tokens_per_segment = None
+split_tokens.passthrough_feature_keys = None
+
+# Parameters for sublayer_call_layer:
+# ==============================================================================
+# None.
+
+# Parameters for sublayer_dropout:
+# ==============================================================================
+sublayer_dropout.dropout_rate = %dropout_rate
+
+# Parameters for sublayer_mask_padding:
+# ==============================================================================
+# None.
+
+# Parameters for sublayer_residual:
+# ==============================================================================
+# None.
+
+# Parameters for sublayer_rms_norm:
+# ==============================================================================
+sublayer_rms_norm.epsilon = 1e-06
+sublayer_rms_norm.name = 'rms_norm'
+
+# Parameters for tpu_estimator_model_fn:
+# ==============================================================================
+tpu_estimator_model_fn.hierarchical_tiling_spec = None
+tpu_estimator_model_fn.init_variable_filter = ''
+tpu_estimator_model_fn.model_info_file = ''
+tpu_estimator_model_fn.outer_batch_size = 1
+tpu_estimator_model_fn.tpu_summaries = False
+
+# Parameters for tpu_mesh_shape:
+# ==============================================================================
+tpu_mesh_shape.ensemble_parallelism = None
+tpu_mesh_shape.model_parallelism = 1
+tpu_mesh_shape.tpu_topology = '4x4'
+
+# Parameters for unit_scaling_convention:
+# ==============================================================================
+unit_scaling_convention.value = False
+
+# Parameters for decoder/Unitransformer:
+# ==============================================================================
+decoder/Unitransformer.d_model = %d_model
+decoder/Unitransformer.ensemble = None
+decoder/Unitransformer.input_full_attention = False
+decoder/Unitransformer.label_smoothing = 0.0
+decoder/Unitransformer.loss_denominator = None
+decoder/Unitransformer.loss_fn = None
+decoder/Unitransformer.loss_on_targets_only = False
+decoder/Unitransformer.max_length = 512
+decoder/Unitransformer.positional_embedding = False
+decoder/Unitransformer.shared_embedding_and_softmax_weights = True
+decoder/Unitransformer.sinusoid_positional_embedding = False
+decoder/Unitransformer.token_dropout_rate = 0.0
+decoder/Unitransformer.vocab_divisor = 128
+decoder/Unitransformer.z_loss = 0.0001
+
+# Parameters for encoder/Unitransformer:
+# ==============================================================================
+encoder/Unitransformer.d_model = %d_model
+encoder/Unitransformer.ensemble = None
+encoder/Unitransformer.input_full_attention = False
+encoder/Unitransformer.label_smoothing = 0.0
+encoder/Unitransformer.loss_denominator = None
+encoder/Unitransformer.loss_fn = None
+encoder/Unitransformer.loss_on_targets_only = False
+encoder/Unitransformer.max_length = 512
+encoder/Unitransformer.positional_embedding = False
+encoder/Unitransformer.shared_embedding_and_softmax_weights = True
+encoder/Unitransformer.sinusoid_positional_embedding = False
+encoder/Unitransformer.token_dropout_rate = 0.0
+encoder/Unitransformer.vocab_divisor = 128
+encoder/Unitransformer.z_loss = 0.0001
+
+# Parameters for unsupervised:
+# ==============================================================================
+unsupervised.preprocessors = \
+[@preprocessors.select_random_chunk,
+@preprocessors.reduce_concat_tokens,
+@preprocessors.split_tokens,
+@preprocessors.denoise]
+
+# Parameters for VarianceScalingInitializer:
+# ==============================================================================
+VarianceScalingInitializer.distribution = 'normal'
+VarianceScalingInitializer.mode = 'fan_in'
+VarianceScalingInitializer.scale = 1.0
+
+# Parameters for VocabEmbedding:
+# ==============================================================================
+VocabEmbedding.scale_variable_like_classifier_weights = False

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b840cd5afdcc806b8175fed5a8800a5aa8be1beb60aab8ab7f650728b122dac2
+size 62321434

special_tokens_map.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "additional_special_tokens": ["<extra_id_0>", "<extra_id_1>", "<extra_id_2>", "<extra_id_3>", "<extra_id_4>", "<extra_id_5>", "<extra_id_6>", "<extra_id_7>", "<extra_id_8>", "<extra_id_9>", "<extra_id_10>", "<extra_id_11>", "<extra_id_12>", "<extra_id_13>", "<extra_id_14>", "<extra_id_15>", "<extra_id_16>", "<extra_id_17>", "<extra_id_18>", "<extra_id_19>", "<extra_id_20>", "<extra_id_21>", "<extra_id_22>", "<extra_id_23>", "<extra_id_24>", "<extra_id_25>", "<extra_id_26>", "<extra_id_27>", "<extra_id_28>", "<extra_id_29>", "<extra_id_30>", "<extra_id_31>", "<extra_id_32>", "<extra_id_33>", "<extra_id_34>", "<extra_id_35>", "<extra_id_36>", "<extra_id_37>", "<extra_id_38>", "<extra_id_39>", "<extra_id_40>", "<extra_id_41>", "<extra_id_42>", "<extra_id_43>", "<extra_id_44>", "<extra_id_45>", "<extra_id_46>", "<extra_id_47>", "<extra_id_48>", "<extra_id_49>", "<extra_id_50>", "<extra_id_51>", "<extra_id_52>", "<extra_id_53>", "<extra_id_54>", "<extra_id_55>", "<extra_id_56>", "<extra_id_57>", "<extra_id_58>", "<extra_id_59>", "<extra_id_60>", "<extra_id_61>", "<extra_id_62>", "<extra_id_63>", "<extra_id_64>", "<extra_id_65>", "<extra_id_66>", "<extra_id_67>", "<extra_id_68>", "<extra_id_69>", "<extra_id_70>", "<extra_id_71>", "<extra_id_72>", "<extra_id_73>", "<extra_id_74>", "<extra_id_75>", "<extra_id_76>", "<extra_id_77>", "<extra_id_78>", "<extra_id_79>", "<extra_id_80>", "<extra_id_81>", "<extra_id_82>", "<extra_id_83>", "<extra_id_84>", "<extra_id_85>", "<extra_id_86>", "<extra_id_87>", "<extra_id_88>", "<extra_id_89>", "<extra_id_90>", "<extra_id_91>", "<extra_id_92>", "<extra_id_93>", "<extra_id_94>", "<extra_id_95>", "<extra_id_96>", "<extra_id_97>", "<extra_id_98>", "<extra_id_99>"]}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d60acb128cf7b7f2536e8f38a5b18a05535c9e14c7a355904270e15b0945ea86
+size 791656

tf_model.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27947b8cf3fc9d1d879752dfafacda9b8bdf6700ba7059a4b946307b534919ea
+size 62473720

tokenizer_config.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids": 100, "additional_special_tokens": ["<extra_id_0>", "<extra_id_1>", "<extra_id_2>", "<extra_id_3>", "<extra_id_4>", "<extra_id_5>", "<extra_id_6>", "<extra_id_7>", "<extra_id_8>", "<extra_id_9>", "<extra_id_10>", "<extra_id_11>", "<extra_id_12>", "<extra_id_13>", "<extra_id_14>", "<extra_id_15>", "<extra_id_16>", "<extra_id_17>", "<extra_id_18>", "<extra_id_19>", "<extra_id_20>", "<extra_id_21>", "<extra_id_22>", "<extra_id_23>", "<extra_id_24>", "<extra_id_25>", "<extra_id_26>", "<extra_id_27>", "<extra_id_28>", "<extra_id_29>", "<extra_id_30>", "<extra_id_31>", "<extra_id_32>", "<extra_id_33>", "<extra_id_34>", "<extra_id_35>", "<extra_id_36>", "<extra_id_37>", "<extra_id_38>", "<extra_id_39>", "<extra_id_40>", "<extra_id_41>", "<extra_id_42>", "<extra_id_43>", "<extra_id_44>", "<extra_id_45>", "<extra_id_46>", "<extra_id_47>", "<extra_id_48>", "<extra_id_49>", "<extra_id_50>", "<extra_id_51>", "<extra_id_52>", "<extra_id_53>", "<extra_id_54>", "<extra_id_55>", "<extra_id_56>", "<extra_id_57>", "<extra_id_58>", "<extra_id_59>", "<extra_id_60>", "<extra_id_61>", "<extra_id_62>", "<extra_id_63>", "<extra_id_64>", "<extra_id_65>", "<extra_id_66>", "<extra_id_67>", "<extra_id_68>", "<extra_id_69>", "<extra_id_70>", "<extra_id_71>", "<extra_id_72>", "<extra_id_73>", "<extra_id_74>", "<extra_id_75>", "<extra_id_76>", "<extra_id_77>", "<extra_id_78>", "<extra_id_79>", "<extra_id_80>", "<extra_id_81>", "<extra_id_82>", "<extra_id_83>", "<extra_id_84>", "<extra_id_85>", "<extra_id_86>", "<extra_id_87>", "<extra_id_88>", "<extra_id_89>", "<extra_id_90>", "<extra_id_91>", "<extra_id_92>", "<extra_id_93>", "<extra_id_94>", "<extra_id_95>", "<extra_id_96>", "<extra_id_97>", "<extra_id_98>", "<extra_id_99>"], "sp_model_kwargs": {}, "name_or_path": "t5-efficient-tiny", "special_tokens_map_file": "t5-efficient-tiny/special_tokens_map.json", "tokenizer_class": "T5Tokenizer"}