<jupyter_start><jupyter_text>Pandas Tutorial---In this tutorial we will take a look at Pandas library which is a very important library when it comes to performing operations on data. We will take a look at things like importing data, extracting data from the table, locating rows and columns, etc. So lets go ahead and see how we implement Pandas. Importing Libraries---<jupyter_code>import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import os<jupyter_output><empty_output><jupyter_text>Pandas Series---Pandas series works like a one-dimensional array and it can hold any type of data such as int, float, string,etc. You can create a series using an array, list or even dictionary.* **Command** - * pd.Series() - This will help us in making our series. We can pass the array in the brackets if we are using an array to form the series, similarly we can pass the data values and index if we are using a dictionary.<jupyter_code>labels = ['a','b','c']
my_data = [10,20,30] 
arr = np.array(my_data) #our array
dic = {'a':10 , 'b':20, 'c':30} #ur dictionary
pd.Series(my_data,labels)
pd.Series(arr,labels) #by using array
pd.Series(dic) #by using dictionary<jupyter_output><empty_output><jupyter_text>Retrieveing Values from a Series---To retrieve any value from a Pandas Series we just need to pass the index of the value that we wish to fetch.<jupyter_code>ser = pd.Series([1,2,3,4] , ['a','b','c','d'])
print(ser)
print(ser['a']) #we used the index a which has the value of 1<jupyter_output>1<jupyter_text>Adding two Pandas Series---Just like string concatenation, we can also add two Pandas Series. Lets do and take a look at it.<jupyter_code>ser1 = pd.Series([1,2,3,4],['a','b','c','d'])
ser2 = pd.Series([1,2,5,4],['a','b','e','d'])
ser3 = ser1 + ser2
print(ser3)<jupyter_output>a    2.0
b    4.0
c    NaN
d    8.0
e    NaN
dtype: float64<jupyter_text>And that is how we can add two Series to generate a third series that holds the sum of first two series. We can see we have Nan values for index 'c' and 'e' and that is because in the first series the values of 'c' is 3 and value of 'e' is 5, now as these two index dont match with any other index in the series hence by default their values are given as NaN.  Data Frames---Most of the times you will find yourself working with Data Frames. It is basically a two dimensional structure that stores the data in tabular format which can be changed and edited as per user's requirements. Just like Series we can make a Data frame as well using array, lists and dictionaries.* **Command** - * pd.DataFrame(dimesnion,rows,columns) = You can pass in arrays and lists that work as index values and the data values and specify the dimensions of the table that you want. You  cam even pass in a series.<jupyter_code>from numpy.random import randn
np.random.seed(101)

df = pd.DataFrame(randn(4,4),['A','B','C','D'],['W','X','Y','Z'])
print(df)<jupyter_output>W         X         Y         Z
A  2.706850  0.628133  0.907969  0.503826
B  0.651118 -0.319318 -0.848077  0.605965
C -2.018168  0.740122  0.528813 -0.589001
D  0.188695 -0.758872 -0.933237  0.955057<jupyter_text>As we can see we have our very own Dataframe. We used A,B,C,D as the rows and W,X,Y,Z as our columns. The random method just generates random number to fill up our DataFrame. We also mentioned dimensions for our DataFrame i.e. 4x4 as we mentioned 4 rows and 4 columns. We can also grab rows and columns and the information in these rows and column if we need to analyse them individually. The columns that we will grab will work as Series<jupyter_code>print(df['W']) #grabbing a single column W
print(df[['W','X']]) #grabbing more than one list by passing in list of columns<jupyter_output>W         X
A  2.706850  0.628133
B  0.651118 -0.319318
C -2.018168  0.740122
D  0.188695 -0.758872<jupyter_text>Adding and Dropping columns in DataFrame---We can make changes to already existing Data Frames by adding or dropping columns. Many times the data that you have might have some columns that are not at all useful for your model so you might have to drop them from the Data Frame in order to make your model more accurate. Lets go ahead and take a look  at how we can add and delete columns in our DataFrame.* **Command** - * df\['column_name'] - To add a new column we can use this command.* df.drop('column_name',axis = 1,inplace = True) - To drop a column from a Data Frame we can use this method. We also define the axis from which we wish to drop, so axis = 1 when we wish to drop a column and axis = 0 when we wish to drop a row. If we dont define the inplace = True, the column wont be dropped permanently from the DataFrame.<jupyter_code>df['new'] = df['X'] + df['Y']
df.head()
df.drop('new',axis = 1,inplace = True)
df.head()<jupyter_output><empty_output><jupyter_text>As you can see, we added a new column to our data frame called 'new'. Then we dropped it using the .drop() method and when we call the head on our Data Frame, we can see that the column does not exist any more in the Data Frame.  Selecting Rows and Columns---In some cases you will need to select some specific rows and columns in order to analyse your data, in such cases method like *'.loc'* and *'.iloc'* come into picture. These will help you grab a row by either using the name of the row or by using the index. To grab a column just pass the column name in your DataFrame Object.* **Command** - * df.loc\['row_name'] -  Helps in fetching the row by using the row name.* df.iloc\[x] - Helps in fetching the row using the Index number(x).* df\[\[column_name]] - Helps in fetching the column using the Column name. Be careful with the two Square Brackets.<jupyter_code>df.loc['C'] #using .loc() 
df.iloc[2] #using .iloc()
df[['X']] #fetching the column
df.loc['B','Y'] #we can pass the row name and column name in .loc to get a specific value<jupyter_output><empty_output><jupyter_text>Suppose you wish to get a subset of your DataFrame, then .loc will help you in doing so. Lets see how.<jupyter_code>df.loc[['A','B'],['W','Y']] #we passed the columns that we need and the rows that we need and we got the info that we needed<jupyter_output><empty_output><jupyter_text>Conditional Selection---Sometimes we will need to retrieve some values from DataFrame based on some condition, this is when conditonal selection comes into picture. Lets see how we can implement it.<jupyter_code>booldf = df > 0
print(df[booldf])<jupyter_output>W         X         Y         Z
A  2.706850  0.628133  0.907969  0.503826
B  0.651118       NaN       NaN  0.605965
C       NaN  0.740122  0.528813       NaN
D  0.188695       NaN       NaN  0.955057<jupyter_text>As we can see, where ever the value was less than 0, we are getting NaN values. We can do this by passing specific columns as well.<jupyter_code>df[df['X'] > 0] #this returns where ever we have values in X column greater than 0<jupyter_output><empty_output><jupyter_text>We can also give multiple Conditions and select columns and rows on that basis. Here's how we do it.<jupyter_code>df[(df['W']>0) & (df['Y']>0)] #condition to check where both W and Y are greater than 0<jupyter_output><empty_output><jupyter_text>Reseting Index---Sometimes you might have alphabets as your index values but you wish to have numeric value instead, so you can just use the *.reset_index()* method and you will have new numerical index values.<jupyter_code>df.reset_index() #this gives us a column with numerical index<jupyter_output><empty_output><jupyter_text>Missing Data---Any data that you get will always have some sort of missing values, its very rare that you get a data without any missing values.In this section we will learn how to deal with the missing values.<jupyter_code>d = {'A':[1,2,np.nan],'B':[5,np.nan,np.nan],'C':[1,2,3]} #creating a new data set
df1 = pd.DataFrame(d)
print(df1)<jupyter_output>A    B  C
0  1.0  5.0  1
1  2.0  NaN  2
2  NaN  NaN  3<jupyter_text>As we can see we have a new data set with NaN values in it and we will be performing all our operations on it. Many a times these NaN values are very less in number and can be discarded. For this purpose we use the *'.dropna()* method. This will drop all the NaN values in our data.<jupyter_code>df1.dropna() #dropped all the rows with NaN values<jupyter_output><empty_output><jupyter_text>Sometimes the amount of NaN values might be huge so in that case we cant just discard them. We will have to fill them up. We use the *.fillna()* method to do so.<jupyter_code>print(df1.fillna(value = 0)) #filling up all the NaN values with 0<jupyter_output>A    B  C
0  1.0  5.0  1
1  2.0  0.0  2
2  0.0  0.0  3<jupyter_text>GroupBy---It is used to group the data present in thee column on basis of the parameters that are provided. Lets see how its actually implemented.* **Command** - * df.groupby('parameter') - We can group our data on basis of some parameter that we need to provide<jupyter_code>#creatinf or new data frame
data = {'Company':['A','A','B','B'],
       'Person':['Sam','Rohit','Yash','Dave'],
       'Sales':[200,120,340,124]}
df2 = pd.DataFrame(data)
print(df2)
comp = df2.groupby('Company') #we grouped our data on basis of Comapnies and then called the mean off of it.
comp.mean()<jupyter_output><empty_output><jupyter_text>we can also use the *.describe()* to get all the values like min,max,mean etc of our DataFrame.<jupyter_code>df2.groupby('Company').describe()<jupyter_output><empty_output><jupyter_text>Operations on DataFrames---* **Commands** - * df.nunique() - Gives us the number of Unique values in a particular column* df.value_counts - Gives us the number of times the unique values occuered in our DataFrame* df.apply() - Can be used to apply some functiont to your data set* df.sort_values() - Sorts the values of a particular column in ascending order that is passed into it<jupyter_code>df2['Company'].nunique() #checking how many companies we have
df2['Person'].value_counts() #checks the number of occurences of each name in that column
#doubles tha no. of sales in each row
def time(x):
    return x*2

df2['Sales'].apply(time)
#sorts the values of the Sales column in Ascending order
df2['Sales'].sort_values()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0942198789298965, 'fraction_numerical': 0.035539933606717436, 'mean_word_length': 4.260914227015922, 'pattern_counts': {'":': 0, '<': 70, '<?xml version=': 0, '>': 74, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '223996', 'license': 'MIT', 'n_tokens_mistral': 3403, 'n_tokens_neox': 3190, 'n_words': 1466, 'path': 'Notebooks/easy_track/pydata/Pandas.ipynb', 'repo_name': 'AM1CODES/onestep-electron-content', 'size': '10242'}