Update README.md

README.md

@@ -6,33 +6,79 @@ model-index:
   results: []
 ---
 
-<!-- This model card has been generated automatically according to the information Keras had access to. You should
-probably proofread and complete it, then remove this comment. -->
+
 
 # pegasus-indonesian-base_finetune
 
-This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
+Github : [PegasusAnthony](https://github.com/nicholaswilven/PEGASUSAnthony/tree/master)
+
+This model is a fine-tuned version of [pegasus-indonesian-base_pretrained](https://huggingface.co/thonyyy/pegasus-indonesian-base_pretrained) on [Indosum](https://paperswithcode.com/dataset/indosum), [Liputan6](https://paperswithcode.com/dataset/liputan6) and [XLSum](https://huggingface.co/datasets/csebuetnlp/xlsum)
 It achieves the following results on the evaluation set:
 - Train Loss: 1.6196
 - Train Accuracy: 0.1079
 - Validation Loss: 1.4097
 - Validation Accuracy: 0.1153
 - Train Lr: 0.00013661868
-- Epoch: 1
+- Epoch: 2
 
-## Model description
+## Intended uses & limitations
 
-More information needed
+This model is uncased, can't read special characters except "," and ".", having hard time understanding numbers, and performance only tested on news article text. 
 
-## Intended uses & limitations
+## Performance
 
-More information needed
+| datasets | rouge-1 | rouge-2 | rouge-L |
+| ---- | ---- | ---- | ---- |
+| Indosum | (TBA) | - | - |
+| Liputan6 | (TBA) | - | - |
+| XLSum | (TBA) | - | - |
 
 ## Training and evaluation data
-
-More information needed
+Finetune dataset:
+1.[Indosum](https://paperswithcode.com/dataset/indosum)
+2.[Liputan6](https://paperswithcode.com/dataset/liputan6)
+3.[XLSum](https://huggingface.co/datasets/csebuetnlp/xlsum)
+
+## Usage
+
+```python
+# Load model and tokenizer
+from transformers import TFPegasusForConditionalGeneration, PegasusTokenizerFast
+model_name = "thonyyy/pegasus-indonesian-base_finetune"
+model = TFPegasusForConditionalGeneration.from_pretrained(model_name)
+tokenizer = PegasusTokenizerFast.from_pretrained(model_name)
+
+# Main function to clean text, removes link, bullet point, non ASCII char, parantheses,
+# punctuation except "," and ".", numbers with dot (enumerating), extra whitespaces, too short sentences.
+import re
+import unicodedata
+def text_cleaning(input_string):
+    lowercase = input_string.lower()
+    remove_link = re.sub(r'(https?:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})([\/\w\.-]*)', '', lowercase).replace("&amp;","&")
+    remove_bullet = "\n".join([T for T in remove_link.split('\n') if '•' not in T and "baca juga:" not in T])
+    remove_accented = unicodedata.normalize('NFKD', remove_bullet).encode('ascii', 'ignore').decode('utf-8', 'ignore')
+    remove_parentheses = re.sub("([\(\|]).*?([\)\|])", "\g<1>\g<2>", remove_accented)
+    remove_punc = re.sub(r"[^\w\d.\s]+",' ', remove_parentheses)
+    remove_num_dot = re.sub(r"(?<=\d)\.|\.(?=\d)|(?<=#)\.","", remove_punc)
+    remove_extra_whitespace =  re.sub(r'^\s*|\s\s*', ' ', remove_num_dot).strip()
+    return ".".join([s for s in remove_extra_whitespace.strip().split('.') if len(s.strip())>10]).replace("_","")
+
+# Article to summarize
+sample_article="""
+Dana Moneter Internasional (IMF) menilai Indonesia telah menunjukkan pemulihan ekonomi yang baik pasca pandemi melalui kinerja makroekonomi yang kuat, didukung penerapan kebijakan moneter dan fiskal secara berhati-hati. Kebijakan forward looking dan sinergi telah berhasil membawa Indonesia menghadapi tantangan global pada tahun 2022 dengan pertumbuhan yang sehat, tekanan inflasi yang menurun, dan sistem keuangan yang stabil.  Bank Indonesia menyambut baik hasil asesmen IMF atas perekonomian Indonesia dalam laporan Article IV Consultation tahun 2023 yang ​dirilis hari ini (26/6).
+Dewan Direktur IMF menyampaikan apresiasi dan catatan positif terhadap berbagai kebijakan yang ditempuh otoritas Indonesia selama tahun 2022. Pertama, keberhasilan otoritas untuk kembali kepada batas maksimal defisit fiskal 3%, lebih cepat dari yang diperkirakan dan komitmen otoritas untuk menerapkan disiplin fiskal. Kedua, penerapan kebijakan moneter yang memadai untuk menjaga stabilitas harga. Ketiga, ketahanan sektor keuangan yang tetap terjaga. Keempat, penerapan UU Cipta Kerja serta UU Pengembangan dan Penguatan Sektor Keuangan, dengan memastikan implementasi yang tepat dan keberlanjutan momentum reformasi untuk mendorong kemudahan berinvestasi, meningkatkan pendalaman pasar keuangan, dan memitigasi dampak scarring dari pandemi. Kelima, strategi diversifikasi Indonesia yang fokus pada upaya hilirisasi dalam rangka meningkatkan nilai tambah ekspor. Keenam, komitmen otoritas untuk mengurangi emisi  gas rumah kaca dan deforestasi.
+"""
+
+# Generate summary
+x = tokenizer(text_cleaning(t), return_tensors = 'tf')
+y = model.generate(**x)
+suummary = tokenizer.batch_decode(y, skip_special_tokens=True)
+print(summary)
+
+```
 
 ## Training procedure
+For replication, go to GitHub page
 
 ### Training hyperparameters
 
@@ -44,8 +90,8 @@ The following hyperparameters were used during training:
 
 | Train Loss | Train Accuracy | Validation Loss | Validation Accuracy | Train Lr      | Epoch |
 |:----------:|:--------------:|:---------------:|:-------------------:|:-------------:|:-----:|
-| 2.3484     | 0.0859         | 1.6304          | 0.1080              | 0.00013661868 | 0     |
-| 1.6196     | 0.1079         | 1.4097          | 0.1153              | 0.00013661868 | 1     |
+| 2.3484     | 0.0859         | 1.6304          | 0.1080              | 0.00013661868 | 1     |
+| 1.6196     | 0.1079         | 1.4097          | 0.1153              | 0.00013661868 | 2     |
 
 
 ### Framework versions
@@ -54,3 +100,6 @@ The following hyperparameters were used during training:
 - TensorFlow 2.12.0
 - Datasets 2.13.1
 - Tokenizers 0.13.3
+
+### Special Thanks
+Research supported with Cloud TPUs from Google’s TPU Research Cloud (TRC)