<jupyter_start><jupyter_text>Dealing with Missing DataAt some point in your dealings with data, you will have to deal with missing values.Depending on the situation, you might need to fill these gaps, or delete rows/columns entirely. Luckily Pandas makes this easy for us.Let’s get our modules and dataset prepared, before we look to delete series or fill gaps.<jupyter_code>import numpy as np
import pandas as pd
#DataFrame is the contract details for our transfer targets, where known.
#'np.nan' is a numpy value that shows that there is not a number.
# In this case, it demonstrates missing data.

df = pd.DataFrame(
 {'Wage':[150000,123000,np.nan] ,'GoalBonus':[4000,np.nan,np.nan],'ImageRights':[50000,70000,100000]
}, index=['Konda','Makho','Grey'], columns=['Wage','GoalBonus','ImageRights']
)
df<jupyter_output><empty_output><jupyter_text>Removing rows & columns with missing dataIf you decide to bin the players with missing data, it is simple with the ‘.dropna()’ method:<jupyter_code>df.dropna()
#drop columns with missing values
df.dropna(axis=1)<jupyter_output><empty_output><jupyter_text>‘.dropna()’ can also take the argument ‘thresh’ to change the amount of missing values you’re happy to deal with. Makho has only 1 missing value, whereas Grey has 2. Below, we’ll allow Makho into our dataset, but continue to exclude Grey:<jupyter_code>df.dropna(thresh=2)<jupyter_output><empty_output><jupyter_text>Fill DataSometimes, deleting rows and columns is a bit drastic. You may instead want to simply fill in the gaps instead. Rather than ‘.dropna()’, we can instead ‘.fillna()’, passing the desired value as the argument.<jupyter_code>df.fillna(value=0)<jupyter_output><empty_output><jupyter_text>You might want to be a bit smarter than filling with 0s. As an example, you might want to take a column and use the average to fill the rest of the gaps:<jupyter_code>df['Wage'].fillna(value=df['Wage'].mean())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09856262833675565, 'fraction_numerical': 0.019507186858316223, 'mean_word_length': 6.245353159851301, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '630483', 'license': 'MIT', 'n_tokens_mistral': 634, 'n_tokens_neox': 595, 'n_words': 233, 'path': 'Data_Analysis_crash_course/Data Analysis Part Two Pandas/Dealing With Missing Data.ipynb', 'repo_name': 'OTeeEnabor/FC_Python_Enabor', 'size': '1948'}