<jupyter_start><jupyter_text>Exploratory data analysis (EDA), using the cleaned data which is the deliverable of the cleaning and cleaning stage.
The EDA, among other things, is to identify the distribution of the target variable ("HasDetections"), so as to determine the appropriate cross validation technique and model evaluation metric.<jupyter_code>#Import the needed modules
import pandas as pd 
import seaborn as sns

#Read the cleaned malware dataset for EDA
cleaned_malware_dataset = pd.read_csv("/content/drive/MyDrive/prediction/cleanedtrain2.csv", low_memory = False)
cleaned_malware_dataset.head()
#Create a chart
chart = sns.countplot(x='HasDetections', data=cleaned_malware_dataset)
chart.set_xlabel("HasDetections", fontsize = 20)
chart.set_ylabel("count", fontsize = 20)<jupyter_output><empty_output><jupyter_text>The values of our target feature ('HasDetections') is equally divided between the two classes, 0 and 1. Based on this, k-fold cross validation will be used as the cross validation technique. The model evaluation metrics would be Accuracy, Precision, Recall and F1.<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07417102966841187, 'fraction_numerical': 0.006980802792321117, 'mean_word_length': 6.4480519480519485, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '181599', 'license': 'MIT', 'n_tokens_mistral': 340, 'n_tokens_neox': 325, 'n_words': 122, 'path': 'MS-malware-suspectibility-detection/1-data-cleaning-EDA/MSmalwareEDA.ipynb', 'repo_name': 'Semiu/malware-detector', 'size': '1146'}