<jupyter_start><jupyter_text>Managing DataFramesBelow the there is code to read in two data frames on Heart health from the UCI repository. Your challenge is to clean and manipulate them using the functions you have learned over the last week and a half. Try to push yourself not to use google on this challenge for as long as possible. This helps to lock in what you've learned and show you that more is sticking than you might think.<jupyter_code>import pandas as pd
headers=['age','sex','cp','trestbps','chol','fbs','restecg','thalach','exang','oldpeak','slope','ca','thal','num']

Cleveland=pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/heart-disease/processed.cleveland.data', names=headers)
Hungarian=pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/heart-disease/processed.hungarian.data', names=headers)<jupyter_output><empty_output><jupyter_text>Summary StatisticsGet summary statistics for each DataFrame, and in markdown below write anything you notice about what is in the data<jupyter_code>print(Cleveland.shape)
pd.set_option('display.max_columns', None)
print(Cleveland.head())

print(Hungarian.shape)
pd.set_option('display.max_columns', None)
print(Hungarian.head())<jupyter_output>(303, 14)
    age  sex   cp  trestbps   chol  fbs  restecg  thalach  exang  oldpeak  \
0  63.0  1.0  1.0     145.0  233.0  1.0      2.0    150.0    0.0      2.3   
1  67.0  1.0  4.0     160.0  286.0  0.0      2.0    108.0    1.0      1.5   
2  67.0  1.0  4.0     120.0  229.0  0.0      2.0    129.0    1.0      2.6   
3  37.0  1.0  3.0     130.0  250.0  0.0      0.0    187.0    0.0      3.5   
4  41.0  0.0  2.0     130.0  204.0  0.0      2.0    172.0    0.0      1.4   

   slope   ca thal  num  
0    3.0  0.0  6.0    0  
1    2.0  3.0  3.0    2  
2    2.0  2.0  7.0    1  
3    3.0  0.0  3.0    0  
4    1.0  0.0  3.0    0  
(294, 14)
   age  sex  cp trestbps chol fbs restecg thalach exang  oldpeak slope ca  \
0   28    1   2      130  132   0       2     185     0      0.0     ?  ?   
1   29    1   2      120  243   0       0     160     0      0.0     ?  ?   
2   29    1   2      140    ?   0       0     170     0      0.0     ?  ?   
3   30    0   1      170  237   0       1     170    [...]<jupyter_text>CleaningWrite a function that takes in a dataframe and cleans any missing values. Missing values should be filled with the mean of the column that they are in for that DataFrame. Apply this function to both DataFrames.<jupyter_code>import numpy as np
print('Cleveland')
print(Cleveland.isnull().sum())
print('Replaced missing values with the mean')
Cleveland1 = Cleveland.replace(0, np.NaN)
Cleveland1.fillna(Cleveland1.mean(), inplace=True)
print(Cleveland1.isnull().sum())

print('Hungarian')
print(Hungarian.isnull().sum())
print('Replaced missing values with the mean')
Hungarian1 = Hungarian.replace(0, np.NaN)
Hungarian1.fillna(Hungarian1.mean(), inplace=True)
print(Hungarian1.isnull().sum())<jupyter_output>Cleveland
age         0
sex         0
cp          0
trestbps    0
chol        0
fbs         0
restecg     0
thalach     0
exang       0
oldpeak     0
slope       0
ca          0
thal        0
num         0
dtype: int64
Replaced missing values with the mean
age         0
sex         0
cp          0
trestbps    0
chol        0
fbs         0
restecg     0
thalach     0
exang       0
oldpeak     0
slope       0
ca          0
thal        0
num         0
dtype: int64
Hungarian
age         0
sex         0
cp          0
trestbps    0
chol        0
fbs         0
restecg     0
thalach     0
exang       0
oldpeak     0
slope       0
ca          0
thal        0
num         0
dtype: int64
Replaced missing values with the mean
age         0
sex         0
cp          0
trestbps    0
chol        0
fbs         0
restecg     0
thalach     0
exang       0
oldpeak     0
slope       0
ca          0
thal        0
num         0
dtype: int64<jupyter_text>JoiningJoin the two DataFrames into one giant DataFrame. Put the Clevland data on top.<jupyter_code>df1 = pd.concat([Cleveland1, Hungarian1], axis=0)
df1
df1.dtypes
import pandas as pd
df1["chol"] = df1.chol.cnovert_objects(convert_numeric=True)<jupyter_output><empty_output><jupyter_text>FilteringFilter the new giant DataFrame so you only have the observations with the highest 25% of cholesterol.<jupyter_code>df1[].max()<jupyter_output><empty_output><jupyter_text>CrosstabsFind categorical data in the new DataFrame and create a crosstab of those features<jupyter_code><jupyter_output><empty_output><jupyter_text>Tidy DataFor this section we are going to use a toy dataset on flights data in the seaborn library.<jupyter_code>import seaborn as sns
flights=sns.load_dataset('flights')<jupyter_output><empty_output><jupyter_text>Is this data in wide or long format? Answer in markdown. Transform it into the other format (if its in long transform it to wide, if its wide transform it to long) using pivot or melt.<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08045746388443017, 'fraction_numerical': 0.0692215088282504, 'mean_word_length': 2.3796610169491523, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '228864', 'license': 'MIT', 'n_tokens_mistral': 1858, 'n_tokens_neox': 1721, 'n_words': 571, 'path': 'Mike_Smith_DataFramesPractice.ipynb', 'repo_name': 'gomlfx/CodeSearchNet', 'size': '4984'}