<jupyter_start><jupyter_text>Road to AI Session 2    Database,Graphs & MathsPANDAS  Importing the library<jupyter_code>import numpy as np
import pandas as pd
print(pd.__version__)<jupyter_output>1.1.3<jupyter_text>Reading the file<jupyter_code>df = pd.read_csv('apy.csv')<jupyter_output><empty_output><jupyter_text>The Basics<jupyter_code>df.head()
df.tail()
#We can see the dimensions of the dataframe using the the shape attribute
df.shape
#We can also extract all the column names as a list
df.columns.tolist()
#function to see statistics like mean, min, etc about each column of the dataset
df.describe()
## max() will show you the maximum values of all columns
df.max()
#get the max value for a particular column
df['Area'].max()
## find the mean of the Production score. 
df['Production'].mean()
## function to identify the row index
df['Production'].argmax()<jupyter_output><empty_output><jupyter_text>**value_counts()** shows how many times each item appears in the column. This particular command shows the number of games in each season<jupyter_code>df['Area'].value_counts()<jupyter_output><empty_output><jupyter_text>Acessing Values<jupyter_code>## get attributes
df.iloc[[df['Production'].argmax()]]
df.iloc[[df['Production'].argmax()]]['Area']<jupyter_output><empty_output><jupyter_text>When you see data displayed in the above format, you're dealing with a Pandas **Series** object, not a dataframe object.<jupyter_code>type(df.iloc[[df['Production'].argmax()]]['Area'])
type(df.iloc[[df['Production'].argmax()]])<jupyter_output><empty_output><jupyter_text>The other really important function in Pandas is the **loc** function. Contrary to iloc, which is an integer based indexing, loc is a "Purely label-location based indexer for selection by label". Since the table are ordered from 0 to 145288, iloc and loc are going to be pretty interchangable in this type of dataset<jupyter_code>df.iloc[:3]
## loc is a "Purely label-location based indexer for selection by label"
df.loc[:3]<jupyter_output><empty_output><jupyter_text>Notice the slight difference in that iloc is exclusive of the second number, while loc is inclusive.  Below is an example of how you can use loc to acheive the same task as we did previously with iloc<jupyter_code>df.loc[df['Production'].argmax(), 'Area']<jupyter_output><empty_output><jupyter_text>A faster version uses the **at()** function. At() is really useful wheneever you know the row label and the column label of the particular value that you want to get.<jupyter_code>df.at[df['Production'].argmax(),  'Area']<jupyter_output><empty_output><jupyter_text>Sorting<jupyter_code>## sort the dataframe in increasing order
df.sort_values('Area').head()
df.groupby('Area')<jupyter_output><empty_output><jupyter_text>Filtering Rows Conditionally<jupyter_code>df[df['Area'] > 500000]
df[(df['Area'] > 5000000) & (df['Area'] < 5555500)]<jupyter_output><empty_output><jupyter_text>Grouping<jupyter_code>##  allows you to group entries by certain attributes
df.groupby('State_Name')['Area'].mean().head()
df.groupby('State_Name')['Area'].value_counts().head(9)
df.values
## Now, you can simply just access elements like you would in an array. 
df.values[0][0]<jupyter_output><empty_output><jupyter_text>Extracting Rows and Columns The bracket indexing operator is one way to extract certain columns from a dataframe.<jupyter_code>df[['Production', 'Area']].head()<jupyter_output><empty_output><jupyter_text>Notice that you can acheive the same result by using the loc function. Loc is a veryyyy versatile function that can help you in a lot of accessing and extracting tasks.<jupyter_code>df.loc[:, ['Production', 'Area']].head()<jupyter_output><empty_output><jupyter_text>Note the difference is the return types when you use brackets and when you use double brackets.<jupyter_code>type(df['Production'])
type(df[['Production']])<jupyter_output><empty_output><jupyter_text>You've seen before that you can access columns through df['col name']. You can access rows by using slicing operations.<jupyter_code>df[0:3]<jupyter_output><empty_output><jupyter_text>Here's an equivalent using iloc<jupyter_code>df.iloc[0:3,:]<jupyter_output><empty_output><jupyter_text>Data Cleaning<jupyter_code>## if there are any missing values in the dataframe, and will then sum up the total for each column
df.isnull().sum()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11662474273953807, 'fraction_numerical': 0.00914703864623828, 'mean_word_length': 7.54296875, 'pattern_counts': {'":': 0, '<': 73, '<?xml version=': 0, '>': 74, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '394246', 'license': 'Apache-2.0', 'n_tokens_mistral': 1447, 'n_tokens_neox': 1390, 'n_words': 428, 'path': 'Phase1/ROAD_TO_AI_S2/pandas/pandas_classroom.ipynb', 'repo_name': 'Amine-Bassoum/RoadToAI-Series', 'size': '4373'}