<jupyter_start><jupyter_text>Hands-On Data Preprocessing in PythonLearn how to effectively prepare data for successful data analytics        AUTHOR: Dr. <NAME>  Chapter 3: Data  Excercise 11)	From 5 colleagues or classmates ask to provide a definition for the term data.     a)	Report these definitions and indicate the similarity among them.     b)	In your own words, define the all-encompassing definition of data put forth in this chapter.    c)	Indicate the two important aspects of the definition in b.    d)	Compare the 5 definitions of data from your colleagues with the all-encompassing definitions and indicate    Excercise 2For this exercise, we are going to use covid_impact_on_airport_traffic.csv. Answer the following questions. This dataset is from Kaggle.com, use this link to see its page: https://www.kaggle.com/terenceshin/covid19s-impact-on-airport-traffic.The key attribute of this dataset is PercentOfBaseline which shows the ratio of air traffic in the specific day compared to pre-pandemic time (1st Feb to 15th March 2020)<jupyter_code>import pandas as pd
pd.read_csv('covid_impact_on_airport_traffic.csv')<jupyter_output><empty_output><jupyter_text>a)	What is the best definition of the data object for this dataset?    b)	Are there any attributes in the data that only have one value? Use .unique() function to check. If there are remove them from the data and update the definition of the data object.    c)	What type of values do the remaining attributes carry?    d)	How much statistical information the attribute ‘PercentOfBaseline’ has? Answer:   Excercise 3    For this exercise, we are going to use US_Accidents.csv. Answer the following questions. This dataset is from Kaggle.com, use this link to see its page: https://www.kaggle.com/sobhanmoosavi/us-accidents.This dataset shows all the car accidents in the US from February 2016 to Dec 2020.<jupyter_code>import pandas as pd
pd.read_csv('US_Accidents.csv')<jupyter_output><empty_output><jupyter_text>a)	What is the best definition of the data object for this dataset?    b)	Are there any attributes in the data that only have one value? Use .unique() function to check. If there are remove them from the data and update the definition of the data object.    c)	What type of values do the remaining attributes carry?    d)	How much statistical information the numerical attributes of the dataset carry?    e)	Compare the statistical information of the numerical attributes and see if any of them are a candidate for data redundancy. Answer:  Excercise 4 For this exercise, we are going to use fatal-police-shootings-data.csv. There are a lot of debates, discussions, dialogues, and protests happening in the US surrounding police killings. The Washington Post has been collecting data on all fatal police shootings in the US. The dataset available to the government and the public alike has date, age, gender, race, location, and other situational information of these fatal police shootings. You can read more about this data on https://www.washingtonpost.com/graphics/investigations/police-shootings-database/, and you can download the last version of the data from https://github.com/washingtonpost/data-police-shootings<jupyter_code>import pandas as pd

pd.read_csv('fatal-police-shootings-data.csv').head()<jupyter_output><empty_output><jupyter_text>a)	What is the best definition of the data object for this dataset?    b)	Are there any attributes in the data that only have one value? Use .unique() function to check. If there are remove them from the data and update the definition of the data object.    c)	What type of values do the remaining attributes carry?    d)	How much statistical information the numerical attributes of the dataset carry?    e)	Compare the statistical information of the numerical attributes and see if any of them are a candidate for data redundancy.  Excercise 5For this exercise, we will be using electricity_prediction.csv. The screenshot below shows the 5 rows of this dataset and a linear regression model created to predict electricity consumption based on the weekday and daily average temperature.<jupyter_code>import pandas as pd

electricity_df = pd.read_csv('electricity_prediction.csv')
electricity_df.head()
from sklearn.linear_model import LinearRegression

X = electricity_df[['Weekday', 'Average Temperature']]
y = electricity_df['Consumption']

lrm = LinearRegression()
lrm.fit(X, y)

print('intercept ', lrm.intercept_)
print(pd.DataFrame({'Predictor': X.columns, 'coefficient': lrm.coef_}))<jupyter_output>intercept  3074181.4950158806
             Predictor   coefficient
0              Weekday -55710.145405
1  Average Temperature  -3476.377056<jupyter_text>The regression model that is derived from the data is presented below.    Consumption = 3074181.5- 55710.1 × Weekday-3476.4 ×Average TemperatureWhat is the fundamental mistake in this analysis? Describe it and provides possible solutions for it.  Answer:  Excercise 6For this exercise, we will be using adult.csv. we used this dataset extensively in chapter 1. Read the dataset using Padans and call it adult_df.<jupyter_code>import pandas as pd

adult_df = pd.read_csv('adult.csv')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05620563974678688, 'fraction_numerical': 0.016880874736236333, 'mean_word_length': 4.799777530589544, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 4, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '158691', 'license': 'MIT', 'n_tokens_mistral': 1477, 'n_tokens_neox': 1361, 'n_words': 671, 'path': 'Chapter03/Excercises Ch3.ipynb', 'repo_name': 'arifmudi/Hands-On-Data-Preprocessing-in-Python-', 'size': '5217'}