license:openrail region:us
---
license: openrail
pretty_name: Finetune1
---
![Change can be sunshine if you let it in..png](https://cdn-uploads.huggingface.co/production/uploads/64c7bfe8ac1016256b69ea02/3-T_BdltbjPE58LEJ3ksN.png)
#  📔 **DATASET** 
| **Dataset**                |       Class                                                           |        Number of Questions         |
|    -------                 |  -----------------------------------------------------------------    |     ------------------------             |
| **FLAN_CoT(zs)**           | Reasoning 、 MATH 、 ScienceQA 、 Commonsense                          |        91910         |
| **Prm800k**                | Reasoning 、 MATH                                                      |        6713         |
| **ScienceQA**              | ScienceQA                                                              |        5177         |
| **SciBench**               | ScienceQA                                                              |         695        | 
| **ReClor**                 | Reasoning                                                              |      1624           | 
| **TheoremQA**              | Commonsense 、 MATH 、 ScienceQA                                        |         800        | 
| **OpenBookQA**             | Text_Understanding 、 Reasoning 、 Commonsense 、 ScienceQA              |        5957         | 
| **ARB**                    | Reasoning 、 MATH 、 ScienceQA 、 Commonsense 、 Text_Understanding       |       605          | 
| **Openassistant-guanaco**  | Commonsense 、 Text_Understanding 、 Reasoning                           |        802         | 
| **SQuAD 2.0**              | Text_Understanding                                                      |        87599         | 
| **CommonsenseQA**          | Commonsense                                                             |      9741           | 
| **Ethics**                 | Commonsense                                                             |     21759            | 
# 📌 **Methon**
## *Dataset Format Definition*
Use "instruction、input、output" tend to lean towards guided datasets. In this format, each sample includes an instruction, an input, and an expected output. The instruction provides guidance on how to process the input to generate the output. This format of dataset is often used to train models to perform specific tasks, as they explicitly indicate the operations the model should perform.
  ```
    {
        "input": "",
        "output": "",
        "instruction": ""
    }
   ```

- ### [FLAN_V2 COT(ZS)](https://huggingface.co/datasets/conceptofmind/cot_submix_original/tree/main)
  We only extract the 'zs_opt' from COT and categorize each task.
- ### [CommonsenseQA](https://huggingface.co/datasets/commonsense_qa)
  We extracted the question and choices from the original CommonsenseQA dataset and placed them in the instruction. We also wrote the input prompt: "Choose A, B, C, D, or E as your solution."
- ### [SQuAD](https://huggingface.co/datasets/squad)
  We used the questions from the SQUAD dataset as instructions and treated the context as the input.
- ### [Ethics](https://huggingface.co/datasets/hendrycks/ethics)
  The ethics dataset, which was originally in labeled format, has been transformed into a true or false format. Additionally, the input now includes the instruction "Give true or false according to ethics."
- ### [OTHER](https://github.com/arielnlee/Platypus/tree/main/data_pipeline)
  Prm800k, ScienceQA, SciBench, ReClor, TheoremQA, OpenBookQA, ARB, and OpenAssistant-Guanaco datasets adopt the same format as Platypus.
## *Sampling Algorithms*
1. First,we are taking all datasets from COT, ARB, TheoremQA and Ethics. ARB and TheoremQA encompass a wide range of fields and have a relatively low total count. Since COT has high quality, we are including the entire dataset. For the Ethics dataset, we are collecting the entire dataset because we want the model to comprehensively learn more about ethics and security aspects.

2. The remaining datasets were initially categorized into the following four groups for the purpose of **Simple Random Sampling**:

    - *Science Questions and Answers* : ScienceQA、SciBench
    - *Reasoning & Mathematics* : ReClor、Prm800k
    - *Text Comprehension* : OpenBookQA、SQuAD
    - *Commonsense* : CommonsenseQA、Openassistant-guanaco
   
   However, we discovered that the total number of datasets in the Science Questions and Answers、Reasoning & Mathematics、and Commonsense categories did not exceed 30,000. As a result, only the Text Comprehension category underwent Simple Random Sampling, while the others were taken in their entirety.
# 🏁 **Feature Work** 

- In the future, we intend to utilize Stratified Sampling due to the imbalance in the number of questions across different datasets, which introduces bias. Conversely, if we opt to randomly sample an equal number of examples from each dataset, it can yield a smaller estimation error for the same total sample size.
- We can even evaluate based on the fine-tuning from the first stage and employ additional scripting techniques to enhance the quality of the dataset.   