The model's interpretability and explanations are crucial for the medical data analysis: the medical diagnosis system must be transparent, understandable, and explainable. Therefore, the doctor and the patient can know how the model makes decisions, which features are important, and how the features affect the model's result [1]}, [2]}. In this section, we mainly introduce the feature importance, permutation importance and SHAP value which can help interpret the model.
