<jupyter_start><jupyter_text>Dealing With Null Data  Introduction:The data have been modified to contain some missing values, identified by NaN.  Using pandas should make this exerciseeasier, in particular for the bonus question.  Step 1: Import the necessary libraries<jupyter_code>import pandas as pd
import numpy as np<jupyter_output><empty_output><jupyter_text>Step 2: Creating the dataset<jupyter_code># Creating date for use as index
dates = pd.date_range(start='2020-01-20',periods= 7)

#creating random data
M = np.random.random((7,7))

# creating data frame
dframe = pd.DataFrame(M, index= dates)

# Give columns name
dframe.columns = ['C1', 'C2', 'C3', 'C4', 'C5', 'C6', 'C7'] 
dframe<jupyter_output><empty_output><jupyter_text>Step 3: Set NaN values in dframe<jupyter_code>dframe.at[0:8,'C7']=np.NaN
dframe.at[0:2,'C6']=np.NaN
dframe.at[5:6,'C5']=np.NaN
dframe<jupyter_output><empty_output><jupyter_text>Step 4: Fill all NULL values with 1020<jupyter_code>dframe= dframe.fillna(1020)
dframe
dframe.at[0:5 , 'C7'] = np.NaN
dframe.at[0:2 , 'C6'] = np.NaN
dframe.at[5:6 , 'C5'] = np.NaN
dframe
# Replace Null values in Column 'C5' with number 123
# Replace Null values in Column 'C6' with number 789
dframe.fillna(value={'C5':123, 'C6':789})
#Replace first NULL value in Column C7 with 789
dframe.fillna(value={'C7':789}, limit=1)<jupyter_output><empty_output><jupyter_text>Step 5: Drop Rows with NULL values<jupyter_code>dframe.dropna()<jupyter_output><empty_output><jupyter_text>Step 6: Drop Columns with NULL values<jupyter_code>dframe.dropna(axis='columns')<jupyter_output><empty_output><jupyter_text>Step 7: Drop Rows with NULL values present in C5 or C6<jupyter_code>dframe.dropna(subset=['C5' ,'C6'])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1329512893982808, 'fraction_numerical': 0.045845272206303724, 'mean_word_length': 7.314285714285714, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '253084', 'license': 'MIT', 'n_tokens_mistral': 706, 'n_tokens_neox': 668, 'n_words': 162, 'path': '05_Dealing_with_NULL_values/Untitled.ipynb', 'repo_name': 'machine-learnig-ai/Pro_Pandas', 'size': '1745'}