Init: 모델 최초 커밋

README.md

@@ -1,3 +1,91 @@
 ---
-license: apache-2.0
+language:
+- ko  # Example: fr
+license: apache-2.0  # Example: apache-2.0 or any license from https://hf.co/docs/hub/repositories-licenses
+library_name: transformers  # Optional. Example: keras or any library from https://github.com/huggingface/hub-docs/blob/main/js/src/lib/interfaces/Libraries.ts
+tags:
+- audio
+- automatic-speech-recognition
+datasets:
+- KsponSpeech
+metrics:
+- wer  # Example: wer. Use metric id from https://hf.co/metrics
 ---
+
+# ko-42maru-wav2vec2-conformer-del-1s
+
+## Table of Contents
+- [ko-42maru-wav2vec2-conformer-del-1s](#ko-42maru-wav2vec2-conformer-del-1s)
+  - [Table of Contents](#table-of-contents)
+  - [Model Details](#model-details)
+  - [Evaluation](#evaluation)
+  - [How to Get Started With the Model](#how-to-get-started-with-the-model)
+
+## Model Details
+- **Model Description:**
+해당 모델은 wav2vec2-conformer base architecture에 scratch pre-training 되었습니다. <br />
+Wav2Vec2ConformerForCTC를 이용하여 KsponSpeech에 대한 Fine-Tuning 모델입니다. <br />
+
+- Dataset use [AIHub KsponSpeech](https://www.aihub.or.kr/aihubdata/data/view.do?currMenu=115&topMenu=100&aihubDataSe=realm&dataSetSn=123) <br />
+Datasets는 해당 Data를 전처리하여 임의로 만들어 사용하였습니다. <br />
+del-1s의 의미는 1초 이하의 데이터 필터링을 의미합니다. <br />
+해당 모델은 **음성전사를 자체 커스텀한 42maru** 기준의 데이터로 학습된 모델입니다. (숫자와 영어는 한글 표기법을 따름) <br />
+
+- **Developed by:**  TADev (@lIlBrother, @ddobokki, @jp42maru)
+- **Language(s):** Korean
+- **License:** apache-2.0
+- **Parent Model:** See the [wav2vec2-conformer](https://huggingface.co/docs/transformers/model_doc/wav2vec2-conformer) for more information about the pre-trained base model. (해당 모델은 wav2vec2-conformer base architecture에 scratch pre-training 되었습니다.)
+
+## Evaluation
+Just using `load_metric("wer")` and `load_metric("wer")` in huggingface `datasets` library <br />
+
+## How to Get Started With the Model
+```python
+from transformers import (
+    AutoConfig,
+    AutoFeatureExtractor,
+    AutoModelForCTC,
+    AutoTokenizer,
+    Wav2Vec2ProcessorWithLM,
+)
+from transformers.pipelines import AutomaticSpeechRecognitionPipeline
+import librosa
+
+# 모델과 토크나이저, 예측을 위한 각 모듈들을 불러옵니다.
+config = AutoConfig.from_pretrained(model_config_path)
+model = AutoModelForCTC.from_pretrained(
+    model_name_or_path,
+    config=config,
+)
+feature_extractor = AutoFeatureExtractor.from_pretrained(model_name_or_path)
+tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
+beamsearch_decoder = build_ctcdecoder(
+    labels=list(tokenizer.encoder.keys()),
+    kenlm_model_path=None,
+)
+processor = Wav2Vec2ProcessorWithLM(
+    feature_extractor=feature_extractor, tokenizer=tokenizer, decoder=beamsearch_decoder
+)
+
+# 실제 예측을 위한 파이프라인에 정의된 모듈들을 삽입.
+asr_pipeline = AutomaticSpeechRecognitionPipeline(
+    model=model,
+    tokenizer=processor.tokenizer,
+    feature_extractor=processor.feature_extractor,
+    decoder=processor.decoder,
+    device=-1,
+)
+
+# 음성파일을 불러오고 beamsearch 파라미터를 특정하여 예측을 수행합니다.
+raw_data, _ = librosa.load(audio_path, sr=16000)
+kwargs = {"decoder_kwargs": {"beam_width": 100}}
+pred = asr_pipeline(inputs=raw_data, **kwargs)["text"]
+# 모델이 자소 분리 유니코드 텍스트로 나오므로, 일반 String으로 변환해줄 필요가 있습니다.
+result = unicodedata.normalize("NFC", pred)
+print(result)
+# 안녕하세요 하나둘셋 테스트입니다.
+```
+*Beam-100 Result (WER)*:
+| "clean" | "other" |
+| ------- | ------- |
+| 21.52   | 25.72   |

config.json

@@ -0,0 +1,112 @@
+{
+  "_name_or_path": "test42/wav2vec2-conformer-del-1s",
+  "activation_dropout": 0.1,
+  "adapter_kernel_size": 3,
+  "adapter_stride": 2,
+  "add_adapter": false,
+  "apply_spec_augment": true,
+  "architectures": [
+    "Wav2Vec2ConformerForCTC"
+  ],
+  "attention_dropout": 0.1,
+  "bos_token_id": 1,
+  "classifier_proj_size": 256,
+  "codevector_dim": 256,
+  "conformer_conv_dropout": 0.1,
+  "contrastive_logits_temperature": 0.1,
+  "conv_bias": false,
+  "conv_depthwise_kernel_size": 31,
+  "conv_dim": [
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512
+  ],
+  "conv_kernel": [
+    10,
+    3,
+    3,
+    3,
+    3,
+    2,
+    2
+  ],
+  "conv_stride": [
+    5,
+    2,
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "ctc_loss_reduction": "mean",
+  "ctc_zero_infinity": true,
+  "diversity_loss_weight": 0.1,
+  "do_stable_layer_norm": true,
+  "eos_token_id": 2,
+  "feat_extract_activation": "gelu",
+  "feat_extract_norm": "layer",
+  "feat_proj_dropout": 0.0,
+  "feat_quantizer_dropout": 0.0,
+  "final_dropout": 0.1,
+  "hidden_act": "gelu",
+  "hidden_dropout": 0.1,
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "layerdrop": 0.0,
+  "mask_feature_length": 64,
+  "mask_feature_min_masks": 0,
+  "mask_feature_prob": 0.05,
+  "mask_time_length": 10,
+  "mask_time_min_masks": 2,
+  "mask_time_prob": 0.05,
+  "max_source_positions": 5000,
+  "model_type": "wav2vec2-conformer",
+  "num_adapter_layers": 3,
+  "num_attention_heads": 12,
+  "num_codevector_groups": 2,
+  "num_codevectors_per_group": 320,
+  "num_conv_pos_embedding_groups": 16,
+  "num_conv_pos_embeddings": 128,
+  "num_feat_extract_layers": 7,
+  "num_hidden_layers": 12,
+  "num_negatives": 100,
+  "output_hidden_size": 768,
+  "pad_token_id": 0,
+  "position_embeddings_type": "relative",
+  "proj_codevector_dim": 256,
+  "rotary_embedding_base": 10000,
+  "tdnn_dilation": [
+    1,
+    2,
+    3,
+    1,
+    1
+  ],
+  "tdnn_dim": [
+    512,
+    512,
+    512,
+    512,
+    1500
+  ],
+  "tdnn_kernel": [
+    5,
+    3,
+    3,
+    1,
+    1
+  ],
+  "torch_dtype": "float32",
+  "transformers_version": "4.20.1",
+  "use_weighted_layer_sum": false,
+  "vocab_size": 72,
+  "xvector_output_dim": 512
+}

preprocessor_config.json

@@ -0,0 +1,10 @@
+{
+  "do_normalize": true,
+  "feature_extractor_type": "Wav2Vec2FeatureExtractor",
+  "feature_size": 1,
+  "padding_side": "right",
+  "padding_value": 0.0,
+  "processor_class": "Wav2Vec2Processor",
+  "return_attention_mask": true,
+  "sampling_rate": 16000
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d2763291f92b8e5e90fbbf27f44f290dcc24cdf5f4ff8ca14596a0e6fa6a0c0f
+size 719408199

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "pad_token": "<pad>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token": "<s>",
+  "do_lower_case": false,
+  "eos_token": "</s>",
+  "name_or_path": "test42/wav2vec2-conformer-del-1s",
+  "pad_token": "<pad>",
+  "processor_class": "Wav2Vec2Processor",
+  "replace_word_delimiter_char": " ",
+  "special_tokens_map_file": null,
+  "tokenizer_class": "Wav2Vec2CTCTokenizer",
+  "unk_token": "<unk>",
+  "word_delimiter_token": "|"
+}

vocab.json

@@ -0,0 +1,74 @@
+{
+    "<pad>": 0,
+    "<unk>": 1,
+    "<s>": 2,
+    "</s>": 3,
+    "|": 4,
+    "\u1171": 5,
+    "\u11b4": 6,
+    "\u1165": 7,
+    "\u11ae": 8,
+    "\u110c": 9,
+    "\u116a": 10,
+    "\u110e": 11,
+    "\u11b3": 12,
+    "\u11bf": 13,
+    "\u116b": 14,
+    "\u11c1": 15,
+    "\u1163": 16,
+    "\u11aa": 17,
+    "\u110d": 18,
+    "\u1173": 19,
+    "\u11ba": 20,
+    "\u1169": 21,
+    "\u1174": 22,
+    "\u1112": 23,
+    "\u11c2": 24,
+    "\u11ab": 25,
+    "\u11b5": 26,
+    "\u1167": 27,
+    "\u11b6": 28,
+    "\u1168": 29,
+    "\u1161": 30,
+    "\u11ad": 31,
+    "\u1170": 32,
+    "\u11bd": 33,
+    "\u11b8": 34,
+    "\u11b1": 35,
+    "\u1109": 36,
+    "\u11bb": 37,
+    "\u11af": 38,
+    "\u116d": 39,
+    "\u1103": 40,
+    "\u11a9": 41,
+    "\u1175": 42,
+    "\u1101": 43,
+    "\u1111": 44,
+    "\u1162": 45,
+    "\u1110": 46,
+    "\u1164": 47,
+    "\u1108": 48,
+    "\u116e": 49,
+    "\u1104": 50,
+    "\u1102": 51,
+    "\u116f": 52,
+    "\u110a": 53,
+    "\u1105": 54,
+    "\u11b7": 55,
+    "\u1106": 56,
+    "\u11b9": 57,
+    "\u116c": 58,
+    "\u1100": 59,
+    "\u11ac": 60,
+    "\u1107": 61,
+    "\u1166": 62,
+    "\u11b0": 63,
+    "\u11bc": 64,
+    "\u11b2": 65,
+    "\u11be": 66,
+    "\u110b": 67,
+    "\u11c0": 68,
+    "\u11a8": 69,
+    "\u110f": 70,
+    "\u1172": 71
+}