1. To make the parsing of each step easier, we few-shot re-generate 15k answers for GSM8k and MATH with an Alpha version of WizardLM 70B model to produce solutions in a step-by-step format, then find out those with a correct answer, and use this data to finetune base Llama model. 2. To enhance the model's ability to adhere to the neural and diverse instructions, we also sample 1.5k open-domain conversations from WizardLM's training data, then merge it with above math corpus as the final SFT training data.