<jupyter_start><jupyter_text>Pandas    1.[Pandas Series](https://pandas.pydata.org/pandas-docs/stable/reference/series.html) resource - https://www.youtube.com/watch?v=m7gxnZx2vT4  1*4=4 points Import pandas and numpy with their aliases<jupyter_code>import numpy as np
import pandas as pd<jupyter_output><empty_output><jupyter_text>Create a variable a = pd.Series([ 100, 200, 300, 400])<jupyter_code>a = pd.Series([ 100, 200, 300, 400])<jupyter_output><empty_output><jupyter_text>Print a, and data type<jupyter_code>print(a,"  ",type(a))<jupyter_output>0    100
1    200
2    300
3    400
dtype: int64    <class 'pandas.core.series.Series'><jupyter_text>Using indexing access the element 300 from the series a.<jupyter_code>a[2]<jupyter_output><empty_output><jupyter_text>What are the values of index for series a?  2*4 = 8 points<jupyter_code>for i in range(len(a)):
  print(i,"  ",a[i])<jupyter_output>0    100
1    200
2    300
3    400<jupyter_text>Change the index to  ['c', 'a', 'b', 'd']<jupyter_code>a = pd.Series([ 100, 200, 300, 400], index=['c', 'a', 'b', 'd'])
a<jupyter_output><empty_output><jupyter_text>Access the value in the series with index 'd'<jupyter_code>a['d']<jupyter_output><empty_output><jupyter_text>Sort the values wrt to the index and print it<jupyter_code>a.sort_index()<jupyter_output><empty_output><jupyter_text>Create a new Pandas Series b having index as 'e', 'f', and 'g' and value 800,450,100 and print it  4*2=8 points<jupyter_code>b= pd.Series({'e':800,'f':450,'g':100})
b<jupyter_output><empty_output><jupyter_text>Append b series at the end of a series<jupyter_code>#print a again after appending b into it
a.append(b)<jupyter_output><empty_output><jupyter_text>Sort the values in descending order of a and print the index of the sorted series<jupyter_code>a= a.sort_values(axis=0,ascending=False)

# print index of a
a.index<jupyter_output><empty_output><jupyter_text>2. [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html)Reference:-https://www.youtube.com/watch?v=KB-19V-cSs4  8*2 = 16 points  Part 1  5 points Create a pandas dataframe df from the series 'a' that we used in the last section, print the dataframe<jupyter_code>df = pd.DataFrame({"0":pd.Series([ 800, 450, 400, 300, 200, 100, 100], index=['e', 'f', 'd', 'b', 'a', 'g', 'c'])})
df
#df.sort_index(axis=0, ascending = False)<jupyter_output><empty_output><jupyter_text>Point to ponder! Don't you think series a and dataframe df looks alike? But there must be some difference between them. Hey buddy think and write your thoughts below.<jupyter_code># happy to know your comment
#Series is 1-D whereas Dataframe is multi dimensional. we can say dataframe as collection/combination of series<jupyter_output><empty_output><jupyter_text>2 + 1*4 = 6 points What is the shape of the datafarme<jupyter_code>df.shape<jupyter_output><empty_output><jupyter_text>Hey! remember shape (7,1) implies dataframe has 7 rows and 1 column.  What is the index of the dataframe, is it same as the series 'a' ?<jupyter_code>df.index<jupyter_output><empty_output><jupyter_text>print the head and tail of the dataframe. Additional - (what does head and tali represent?)<jupyter_code>#head
df.head()
#tail
df.tail()<jupyter_output><empty_output><jupyter_text>Rename the columns of the dataframe as 'points'  2 points<jupyter_code>df.columns = ['points']
df<jupyter_output><empty_output><jupyter_text>Create another Series 'fruits', which contains random names of fruits from ['orange','mango','apple']. The series should contain 7 elements, randomly selected from ['orange','mango','apple']  5 points<jupyter_code>#Create fruits array
fruits = np.array(['orange', 'orange', 'mango', 'mango', 'mango', 'mango', 'mango'])
fruits<jupyter_output><empty_output><jupyter_text>The above output might vary based randomness<jupyter_code>#Create series fruits out of fruits array
pd.Series(fruits)<jupyter_output><empty_output><jupyter_text>Change the index of fruits to the index of dataframe df  5 points<jupyter_code>pd.Series(fruits, index = df.index)<jupyter_output><empty_output><jupyter_text>Add this fruits series as a new column to the dataframe df with its column name as 'fruits'  print the head of the dataframe to verify<jupyter_code>df['fruits'] = pd.Series(fruits, index = df.index)
df.head()<jupyter_output><empty_output><jupyter_text>Bonus optional question: What if we had to changed the index of the dataframe? Try to add any series to the dataframe with the origianl index (when index of series is not same as to the index of dataframe)Use series as pd.Series([100,200,300,400,500,600,700]) and try to add it to df with column name Bonus_number.Bonus marks: 5 points<jupyter_code>#bonus code here
df['Bonus_number'] = pd.Series([100,200,300,400,500,600,700]) 
df<jupyter_output><empty_output><jupyter_text>Part 2Reference:- https://www.youtube.com/watch?v=yPVQZZmheCg Create a dataframe  d1 where the columns are  ‘city’ : [‘Chandigarh’, ‘Delhi’, ‘Kanpur’, ‘Chennai’, ‘Manali’ ] and ‘Temperature’ : [15, 22, 20, 26,-2]   2*5=10 points<jupyter_code>d1 = pd.DataFrame({'city' : ['Chandigarh', 'Delhi', 'Kanpur', 'Chennai', 'Manali' ],'Temperature' : [15, 22, 20, 26,-2]})<jupyter_output><empty_output><jupyter_text>Print d1<jupyter_code>d1<jupyter_output><empty_output><jupyter_text>What is the shape of d1.<jupyter_code>d1.shape<jupyter_output><empty_output><jupyter_text>Set city = d1['city']<jupyter_code>city = d1['city']<jupyter_output><empty_output><jupyter_text>print city What is the type of city.<jupyter_code>city<jupyter_output><empty_output><jupyter_text>Create another dataframe d2 where the columns are 'city' - ['Bengalaru','Coimbatore','Srirangam','Pondicherry'] 'Temperature' - [24,35,36,39]  3+2+5 = 10 points<jupyter_code>d2 = pd.DataFrame({'city':['Bengalaru','Coimbatore','Srirangam','Pondicherry'],'Temperature':[24,35,36,39]})
d2<jupyter_output><empty_output><jupyter_text>print the shape of this dataframe<jupyter_code>d2.shape<jupyter_output><empty_output><jupyter_text>merge the two dataframes together, save it in a new dataframe named 'd3'<jupyter_code>d3 = pd.concat([d1,d2])
d3<jupyter_output><empty_output><jupyter_text>fun fact at high level1. .concat() simply stacks multiple DataFrame together either vertically, or stitches horizontally after aligning on index2. .merge() first aligns two DataFrame' selected common column(s) or index, and then pick up the remaining columns from the aligned rows of each DataFrameDo you know about join function? This interesting quest we give to you to find.  Select the part of the dataframe such that it contains cities where temp is less then or equal to 20 How many cities are there?   5+5=10 points<jupyter_code>d3[d3['Temperature'] <= 20]<jupyter_output><empty_output><jupyter_text>Select the part of the dataframe such that it contains the cities where tempearature greater than or equal to 35<jupyter_code>d3[d3['Temperature'] >= 35]<jupyter_output><empty_output><jupyter_text>3. Applying functions to columns and creating new columnsReference: https://www.youtube.com/watch?v=L0wMml__Gow  20 points We need to create another column in d3, which contains  a boolean value for each city to indicate whether it's a union territory or not. - HINT: Chandigarh, Pondicherry and Delhi are only 3 union territories here.<jupyter_code># write function here
d3['is_ut'] = d3['city'].apply(lambda x: x in ['Chandigarh','Pondicherry','Delhi'])
#print d3
d3<jupyter_output><empty_output><jupyter_text>The temperatures mentioned in ‘Temperature’ column are mentioned in Celsius, we need another column which contains the same in Fahrenheit.  HINT - - Define a function c_to_f which takes input temp in celsius and returns a value with temperature in Fahrenheit.- To check: c_to_f(10) should return 50.<jupyter_code># write function here
def c_to_f(n):
  return n * 9/5 + 32
# check  function c_to_f(10)
c_to_f(10)
# apply function c_to_f to d3 to create a column 'temp_farenhiet'
d3['temp_farenhiet'] = d3['Temperature'].apply(lambda x: c_to_f(x))
d3<jupyter_output><empty_output><jupyter_text>4. Indexing and selecting rows in DataFrameReference:-https://www.youtube.com/watch?v=XKFRFDgA7ZY  20 points Select subset of the dataframe d3 such that it contains the cities which are union territories.<jupyter_code>d3[d3['is_ut'] == True]<jupyter_output><empty_output><jupyter_text>Select a subset of the dataframe d3 such that it contains the cities which only have temperature above 90 Farenhiet.<jupyter_code>d3[d3['temp_farenhiet'] > 90]<jupyter_output><empty_output><jupyter_text>Select only the first three rows of the dataframe d3.<jupyter_code>d3.iloc[:3]<jupyter_output><empty_output><jupyter_text>Select all the rows and last two columns in the dataframe.<jupyter_code>d3.iloc[:,-2:]<jupyter_output><empty_output><jupyter_text>5. Reading csv file and group byReference: https://www.youtube.com/watch?v=ENhGz1HkzvY https://www.youtube.com/watch?v=nulx_VmV7dE  2*2=4 points Read the file 'weather_data.csv' and store it in a pandas dataframe 'df'<jupyter_code>df = pd.read_csv('https://raw.githubusercontent.com/Sagar-Gilda/Python/main/weather_data.csv')<jupyter_output><empty_output><jupyter_text>print the shape and head of the dataframe  New Section<jupyter_code>print(df.shape)
print(df.head())<jupyter_output>(6, 4)
        day  temperature  windspeed  event
0  1/1/2017           32          6   Rain
1  1/2/2017           35          7  Sunny
2  1/3/2017           28          2   Snow
3  1/4/2017           24          7   Snow
4  1/5/2017           32          4   Rain<jupyter_text>Group the dataframe wrt to 'event' and save the resulting output in a variable named 'g'  5*3 = 15 points<jupyter_code>g = df.groupby('event')<jupyter_output><empty_output><jupyter_text>Print g and  type(g)<jupyter_code>print(g)
type(g)<jupyter_output><pandas.core.groupby.generic.DataFrameGroupBy object at 0x7f6b0ec0dc10><jupyter_text>Iterate over all the temperatures in g and print them<jupyter_code>list(g)<jupyter_output><empty_output><jupyter_text>Use the describle method on g to get the complete description on it.<jupyter_code>g.describe()<jupyter_output><empty_output><jupyter_text>Create a new column in the dataframe df, named 'hot_temp' which contains True ---if temp is above 30, else False  5 points<jupyter_code>#code here to filter df with above condition
df['hot_temp'] = df['temperature'].apply(lambda x: x > 30)
# print df
df<jupyter_output><empty_output><jupyter_text>Data RangeReference:-https://www.youtube.com/watch?v=dRxCvSbMEto  5*3 = 15 points Create a pandas daterange where starting date is 1st of January,2020 and end date is 1st of April 2021, store it in a new variable named 'a'<jupyter_code>a = pd.date_range(start='2020-01-01',end='2021-04-01')<jupyter_output><empty_output><jupyter_text>print a<jupyter_code>a<jupyter_output><empty_output><jupyter_text>What is the len of a?<jupyter_code>len(a)<jupyter_output><empty_output><jupyter_text>What is the type of a?<jupyter_code>type(a)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11488321564367192, 'fraction_numerical': 0.038294405214557306, 'mean_word_length': 6.419073203492276, 'pattern_counts': {'":': 1, '<': 196, '<?xml version=': 0, '>': 198, 'https://': 11, 'lorem ipsum': 0, 'www.': 8, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '49', 'dir': 'jupyter-structured-clean-dedup', 'id': '614211', 'license': 'MIT', 'n_tokens_mistral': 4191, 'n_tokens_neox': 3909, 'n_words': 1060, 'path': '4_Pandas_output.ipynb', 'repo_name': 'Sagar-Gilda/Python', 'size': '11046'}