<jupyter_start><jupyter_text>Day 6 : Data Loading and Manipulation and Visulatiozation  You can use the following liberaries for your assignment:> Numpy, Pandas, Matplotlib, Seaborn, LASIO, Welly  Kindly load the las file of Exported.csv file from the data folder Perform the below Tasks:>1. Investigate the component of the data file (number of columns , numbers of observations, Null values, normal statistics) 2. Plot the null values as bars3. Create a copy of the data frame and drop the NAN values3. Use the other copy to fill-in NAN values.4. Which option do you prefer to work with regarding the relationship with PHIE and DT or PHIE and RHOB<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sb
import lasio
import welly

from IPython.display import display
df = pd.read_csv("C:/Users/HP/Documents/GitHub/GeoML-2.0/10DaysChallenge/Dutch_F3_Logs/Exported.csv", delimiter=',')
display(df)
#1) Data Investigation

display('Col, Rows',df.shape)
display('info',df.info())
display('Null values',df.isna().sum())
display('Null values%',df.isna().sum()/ df.shape[0] *100)
display('normal statistics',df.describe())
#2) Plot the null values as bars
df.isnull().sum().plot(kind = 'bar', title = 'Missing Values Comparison', figsize=(20,6))
#3) Create a copy of the data frame and drop the NAN values
df_dropna=df.copy()
print('original df =',df_dropna.shape)
df_dropna.dropna(inplace=True)
print('after removing nans =',df_dropna.shape)
print('=' *100)
print('This dataset has {0} rows and {1} columns'.format(df_dropna.shape[0],df_dropna.shape[1]))
#3) Use the other copy to fill-in NAN values (interpolation).
df_fillna=df.copy()
df_fillna=df_fillna.interpolate(method ='linear', limit_direction ='backward')
display('after filling nans',df_fillna)
#4) PHIE vs. DT comparison
plt.figure(figsize=(10,5))
plt.scatter(df_dropna.PHIE, df_dropna.DT)
plt.title('PHIE vs DT plot after removing null values ')
plt.xlabel('PHIE')
plt.ylabel('DT')
plt.grid()
plt.show();

plt.figure(figsize=(10,5))
plt.scatter(df_fillna.PHIE, df.DT)
plt.title('PHIE vs DT plot after filling null values using interpolation')
plt.xlabel('PHIE')
plt.ylabel('DT')
plt.grid()
plt.show();
#4) PHIE vs. RHOB comparison
plt.figure(figsize=(10,5))
plt.scatter(df_dropna.PHIE, df_dropna.RHOB)
plt.title('PHIE vs RHOB plot after removing null values ')
plt.xlabel('PHIE')
plt.ylabel('RHOB')
plt.grid()
plt.show();

plt.figure(figsize=(10,5))
plt.scatter(df_fillna.PHIE, df.RHOB)
plt.title('PHIE vs RHOB plot after filling null values using interpolation')
plt.xlabel('PHIE')
plt.ylabel('RHOB')
plt.grid()
plt.show();<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13373403456048083, 'fraction_numerical': 0.016153268219383923, 'mean_word_length': 6.063660477453581, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '500385', 'license': 'MIT', 'n_tokens_mistral': 983, 'n_tokens_neox': 921, 'n_words': 281, 'path': 'RY Day6 of 10.ipynb', 'repo_name': 'RehabYousry/10Days_Challenge', 'size': '2662'}