Update README.md

README.md

@@ -17,15 +17,15 @@ base_model:
 
 ## Model Information
 
-This model is a continually pre-trained version of the [meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) architecture, fine-tuned on extensive Bangla datasets. The primary goal of the continual pretraining was to enhance the model's ability to generate high-quality Bangla text. By extending the pretraining process specifically on Bangla data, the model has demonstrated superior performance in tasks related to Bangla language understanding evaluation benchmarks and text generation.
+This model is a continually pre-trained version of the [meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) architecture, fine-tuned on extensive Bangla datasets. The primary goal of the continual pretraining was to enhance the model's ability to generate high-quality Bangla text. By extending the pretraining process specifically on Bangla data, the model has demonstrated superior performance in Bangla language understanding evaluation benchmarks and text generation tasks.
 
-**Model Architecture:** Llama 3.2 is an auto-regressive language model that uses an optimized transformer architecture. 
+**Model Architecture:** Llama 3.2 is an auto-regressive language model with optimized transformer architecture. 
 
 |  | Training Data | Params | Input modalities | Output modalities | Context Length | GQA | Shared Embeddings | Token count | Knowledge cutoff |
 | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- |
 | Llama 3.2 (text only)  | Hishab curated Bangla text corpus | 3B(3.21B) | Monolingual Text(Bangla) | Monolingual Text(Bangla)  | 4096 | Yes | Yes | 6B tokens | |
 
-**Supported Languages:** Bengali(primary) and English(secondary)
+**Supported Languages:** Bengali (primary) and English (secondary)
 
 **Llama 3.2 Model Family:** Token counts refer to pretraining data only. All model versions use Grouped-Query Attention (GQA) for improved inference scalability.
 
@@ -33,7 +33,7 @@ This model is a continually pre-trained version of the [meta-llama/Llama-3.2-3B]
 
 **Status:** This is a static model trained on an offline dataset. Future versions may be released to improve model capabilities.
 
-**License:** We are using a similar license of Llama 3.2. Use of Llama 3.2 is governed by the [Llama 3.2 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE) (a custom, commercial license agreement).
+**License:** We are using a similar license to Llama 3.2. Use of Llama 3.2 is governed by the [Llama 3.2 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE) (a custom, commercial license agreement).
 
 
 ## How to use
@@ -66,17 +66,17 @@ pipe("আমাদের দেশের নাম")
 
 ## Training Data
 
-**Overview:** We have collected a large Bangla raw dataset of text data from a wide variety of sources. Our collected data so far includes a mix of web documents, books, translated text, transliterated text, transcribe text, code-mixed text, conversations, and open sources raw data. The dataset is cleaned and filtered by different filtering criteria to ensure the quality of the data. Our collected data size roughly around 268 GB. We separated __22GB__ data from that using a ratio of the data actual data size. Total trained tokens are __6B__ tokens.
+**Overview:** We have collected a large Bangla raw dataset of text data from a wide variety of sources. Our collected data so far includes a mix of web documents, books, translated text, transliterated text, transcribe text, code-mixed text, conversations, and open-source raw data. The dataset is cleaned and filtered by different filtering criteria to ensure the quality of the data. Our collected data size is roughly around 268 GB. We separated __22GB__ data from that using a ratio of the data actual data size. Total trained tokens are __6B__ tokens.
 
 Data sources summary:
-- Web documents: Extract, clean, and filter common crawl data
-- Books: Extract, clean, and filter book data
+- Web documents: Extracted, clean, and filtered common crawl data
+- Books: Extracted, clean, filtered books data
 - Transcribed text: Used in-house Bangla ASR model to transcribe Bangla audio data
-- Translation data: We trained a Bangla-English translation LLM model and used it to translate English data to Bangla
-- Code-mixed data: We trained a Bangla-English code-mixed LLM model and used it to generate code-mixed data
+- Translation data: We trained an English-Bangla translation LLM model and used it to translate English data to Bangla
+- Code-mixed data: We trained an English-Bangla code-mixed LLM model and used it to generate code-mixed data
 - Transliteration data: We trained a Bangla-English transliteration LLM model and used it to generate transliterated data
 - Synthetic data: We generated synthetic data using a Bangla LLM model
-- Others: We scrap some selected website data, used open-source data, and used some other data sources
+- Others: We scrapped some selected website data, used open-source data, and used some other data sources
 
 
 ## Benchmarks
@@ -96,10 +96,10 @@ We evaluated the models on the following datasets:
 
 #### English Benchmark datasets
 - [MMLU](https://huggingface.co/datasets/cais/mmlu): This is a massive multitask test consisting of multiple-choice questions from various branches of knowledge. 
-- [CommonseQa](https://huggingface.co/datasets/tau/commonsense_qa): CommonsenseQA is a new multiple-choice question-answering dataset that requires different types of commonsense knowledge to predict the correct answers .
+- [CommonseQa](https://huggingface.co/datasets/tau/commonsense_qa): CommonsenseQA is a new multiple-choice question-answering dataset that requires different types of commonsense knowledge to predict the correct answers.
 - [OpenbookQA](https://huggingface.co/datasets/allenai/openbookqa): OpenBookQA aims to promote research in advanced question-answering, probing a deeper understanding of both the topic (with salient facts summarized as an open book, also provided with the dataset) and the language it is expressed in.
 - [Piqa](https://huggingface.co/datasets/ybisk/piqa): The PIQA dataset focuses on physical commonsense reasoning, challenging AI to handle everyday situations requiring practical knowledge and unconventional solutions. Inspired by instructables.com, it aims to enhance AI's ability to understand and reason about physical interactions.
-- [BoolQ](https://huggingface.co/datasets/google/boolq): BoolQ is a question answering dataset for yes/no questions containing 15942 examples. These questions are naturally occurring ---they are generated in unprompted and unconstrained settings. Each example is a triplet of (question, passage, answer), with the title of the page as optional additional context. The text-pair classification setup is similar to existing natural language inference tasks.
+- [BoolQ](https://huggingface.co/datasets/google/boolq): BoolQ is a question-answer dataset for yes/no questions containing 15942 examples. These questions are naturally occurring. They are generated in unprompted and unconstrained settings. Each example is a triplet of (question, passage, answer), with the title of the page as optional additional context. The text-pair classification setup is similar to existing natural language inference tasks.
 
 ### Evaluation Results
 
@@ -121,8 +121,8 @@ We evaluated the models on the following datasets:
 
 | Model                                | Shots  | MMLU         | BoolQ      | Commonsense QA     | OpenBook QA     | PIQA      |
 |--------------------------------------|--------|--------------|------------|--------------------|-----------------|-----------|
-| llama-3.2-3b                         | 0-shot | **0.541**    | **0.729**  | **0.643**          | **0.430**       | **0.774** |
-|                                      | 5-shot | **0.563**    | **0.735**  | **0.666**          | **0.454**       | **0.796** |
+| llama-3.2-3b                         | 0-shot | **0.54**    | **0.73**  | **0.64**          | **0.43**       | **0.77** |
+|                                      | 5-shot | **0.56**    | **0.73**  | **0.67**          | **0.45**       | **0.80** |
 | titulm-llama-3.2-3b-v1.0             | 0-shot | 0.47         | 0.70       | 0.58               | 0.39            | 0.76      |
 |                                      | 5-shot | 0.53         | 0.70       | 0.63               | 0.44            | 0.78      |