<jupyter_start><jupyter_text>Questions  1. Pandas Series Create a series using list = [6,7,8,9,2,3,4,5] and print the output series as the square of each number in the list.Hint: If input series = 1,2,3 the output series should be 1,4,25Hint: First create the series and then using apply and lambda find the output series.<jupyter_code>import numpy as np
import pandas as pd
series_1 = pd.Series([6,7,8,9,2,3,4,5])
series_2 = series_1 ** 2
print(series_1)
print(series_2)<jupyter_output>0    6
1    7
2    8
3    9
4    2
5    3
6    4
7    5
dtype: int64
0    36
1    49
2    64
3    81
4     4
5     9
6    16
7    25
dtype: int64<jupyter_text>2. DataFrames Given a dataframe 'd1' use the following commands and analyse the result.describe() columnsshape<jupyter_code>import numpy as np
import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
print(df.describe())
print(df.columns)
print(df.shape)<jupyter_output>X           Y        FFMC         DMC          DC         ISI  \
count  517.000000  517.000000  517.000000  517.000000  517.000000  517.000000   
mean     4.669246    4.299807   90.644681  110.872340  547.940039    9.021663   
std      2.313778    1.229900    5.520111   64.046482  248.066192    4.559477   
min      1.000000    2.000000   18.700000    1.100000    7.900000    0.000000   
25%      3.000000    4.000000   90.200000   68.600000  437.700000    6.500000   
50%      4.000000    4.000000   91.600000  108.300000  664.200000    8.400000   
75%      7.000000    5.000000   92.900000  142.400000  713.900000   10.800000   
max      9.000000    9.000000   96.200000  291.300000  860.600000   56.100000   

             temp          RH        wind        rain         area  
count  517.000000  517.000000  517.000000  517.000000   517.000000  
mean    18.889168   44.288201    4.017602    0.021663    12.847292  
std      5.806625   16.317469    1.791653    0.295959    63.655[...]<jupyter_text>3. Set Index in Dataframe Using set_index command set the column 'X' as the index of the dataset and then print the head of the dataset.Hint: Use inplace = False<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
df_2 = df.set_index('X', inplace=False)
print(df_2.head())<jupyter_output>Y month  day  FFMC   DMC     DC  ISI  temp  RH  wind  rain  area
X                                                                  
7  5   mar  fri  86.2  26.2   94.3  5.1   8.2  51   6.7   0.0   0.0
7  4   oct  tue  90.6  35.4  669.1  6.7  18.0  33   0.9   0.0   0.0
7  4   oct  sat  90.6  43.7  686.9  6.7  14.6  33   1.3   0.0   0.0
8  6   mar  fri  91.7  33.3   77.5  9.0   8.3  97   4.0   0.2   0.0
8  6   mar  sun  89.3  51.3  102.2  9.6  11.4  99   1.8   0.0   0.0<jupyter_text>4. Sorting Dataframes Sort the dataframe on 'month' and 'day' in ascending order in the dataframe 'df'.<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
df_2 = df.sort_values(by=['month', 'day'], ascending = True)
print(df_2.head(20))<jupyter_output>X  Y month  day  FFMC    DMC     DC   ISI  temp  RH  wind  rain   area
241  4  4   apr  fri  83.0   23.3   85.3   2.3  16.7  20   3.1   0.0   0.00
442  6  5   apr  mon  87.9   24.9   41.6   3.7  10.9  64   3.1   0.0   3.35
19   6  4   apr  sat  86.3   27.4   97.1   5.1   9.3  44   4.5   0.0   0.00
239  7  5   apr  sun  81.9    3.0    7.9   3.5  13.4  75   1.8   0.0   0.00
469  6  3   apr  sun  91.0   14.6   25.6  12.3  13.7  33   9.4   0.0  61.13
470  5  4   apr  sun  91.0   14.6   25.6  12.3  17.6  27   5.8   0.0   0.00
176  6  5   apr  thu  81.5    9.1   55.2   2.7   5.8  54   5.8   0.0   4.61
196  6  5   apr  thu  81.5    9.1   55.2   2.7   5.8  54   5.8   0.0  10.93
240  6  3   apr  wed  88.0   17.2   43.5   3.8  15.2  51   2.7   0.0   0.00
12   6  5   aug  fri  63.5   70.8  665.3   0.8  17.0  72   6.7   0.0   0.00
78   1  2   aug  fri  90.1  108.0  529.8  12.5  14.7  66   2.7   0.0   0.00
142  8  6   aug  fri  90.1  108.0  529.8  12.5  21.2  51   8.9   0.0   0.61
184  8  6   [...]<jupyter_text>5. Indexing Dataframes Print only the even numbers of rows of the dataframe 'df'.<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
df_2 = df[2::2]
print(df_2.head(20))<jupyter_output>X  Y month  day  FFMC    DMC     DC   ISI  temp  RH  wind  rain  area
2   7  4   oct  sat  90.6   43.7  686.9   6.7  14.6  33   1.3   0.0   0.0
4   8  6   mar  sun  89.3   51.3  102.2   9.6  11.4  99   1.8   0.0   0.0
6   8  6   aug  mon  92.3   88.9  495.6   8.5  24.1  27   3.1   0.0   0.0
8   8  6   sep  tue  91.0  129.5  692.6   7.0  13.1  63   5.4   0.0   0.0
10  7  5   sep  sat  92.5   88.0  698.6   7.1  17.8  51   7.2   0.0   0.0
12  6  5   aug  fri  63.5   70.8  665.3   0.8  17.0  72   6.7   0.0   0.0
14  6  5   sep  wed  92.9  133.3  699.6   9.2  26.4  21   4.5   0.0   0.0
16  5  5   mar  sat  91.7   35.8   80.8   7.8  15.1  27   5.4   0.0   0.0
18  6  4   mar  wed  89.2   27.9   70.8   6.3  15.9  35   4.0   0.0   0.0
20  6  4   sep  tue  91.0  129.5  692.6   7.0  18.3  40   2.7   0.0   0.0
22  7  4   jun  sun  94.3   96.3  200.0  56.1  21.0  44   4.5   0.0   0.0
24  7  4   aug  sat  93.5  139.4  594.2  20.3  23.7  32   5.8   0.0   0.0
26  7  4   sep  fri  92.4  117.9  668.[...]<jupyter_text>6. Selecting Columns of a Dataframe Print out the columns 'month', 'day', 'temp', 'area' from the dataframe 'df'.<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
df_2 = df[['month', 'day', 'temp', 'area']]
print(df_2.head(20))<jupyter_output>month  day  temp  area
0    mar  fri   8.2   0.0
1    oct  tue  18.0   0.0
2    oct  sat  14.6   0.0
3    mar  fri   8.3   0.0
4    mar  sun  11.4   0.0
5    aug  sun  22.2   0.0
6    aug  mon  24.1   0.0
7    aug  mon   8.0   0.0
8    sep  tue  13.1   0.0
9    sep  sat  22.8   0.0
10   sep  sat  17.8   0.0
11   sep  sat  19.3   0.0
12   aug  fri  17.0   0.0
13   sep  mon  21.3   0.0
14   sep  wed  26.4   0.0
15   sep  fri  22.9   0.0
16   mar  sat  15.1   0.0
17   oct  mon  16.7   0.0
18   mar  wed  15.9   0.0
19   apr  sat   9.3   0.0<jupyter_text>7. Dataframe iloc Using iloc index the dataframe to print all the rows of the columns at index 3,4,5.Hint: Use 3,4,5 not 2,3,4<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
df_2 = df.iloc[:,3:6]
print(df_2.head(20))<jupyter_output>day  FFMC    DMC
0   fri  86.2   26.2
1   tue  90.6   35.4
2   sat  90.6   43.7
3   fri  91.7   33.3
4   sun  89.3   51.3
5   sun  92.3   85.3
6   mon  92.3   88.9
7   mon  91.5  145.4
8   tue  91.0  129.5
9   sat  92.5   88.0
10  sat  92.5   88.0
11  sat  92.8   73.2
12  fri  63.5   70.8
13  mon  90.9  126.5
14  wed  92.9  133.3
15  fri  93.3  141.2
16  sat  91.7   35.8
17  mon  84.9   32.8
18  wed  89.2   27.9
19  sat  86.3   27.4<jupyter_text>8. Dataframes loc Using loc function print out all the columns and rows from 2 to 20 of the 'df' dataset.<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
df_2 = df.loc[2:20,:]
print(df_2)<jupyter_output>X  Y month  day  FFMC    DMC     DC   ISI  temp  RH  wind  rain  area
2   7  4   oct  sat  90.6   43.7  686.9   6.7  14.6  33   1.3   0.0   0.0
3   8  6   mar  fri  91.7   33.3   77.5   9.0   8.3  97   4.0   0.2   0.0
4   8  6   mar  sun  89.3   51.3  102.2   9.6  11.4  99   1.8   0.0   0.0
5   8  6   aug  sun  92.3   85.3  488.0  14.7  22.2  29   5.4   0.0   0.0
6   8  6   aug  mon  92.3   88.9  495.6   8.5  24.1  27   3.1   0.0   0.0
7   8  6   aug  mon  91.5  145.4  608.2  10.7   8.0  86   2.2   0.0   0.0
8   8  6   sep  tue  91.0  129.5  692.6   7.0  13.1  63   5.4   0.0   0.0
9   7  5   sep  sat  92.5   88.0  698.6   7.1  22.8  40   4.0   0.0   0.0
10  7  5   sep  sat  92.5   88.0  698.6   7.1  17.8  51   7.2   0.0   0.0
11  7  5   sep  sat  92.8   73.2  713.0  22.6  19.3  38   4.0   0.0   0.0
12  6  5   aug  fri  63.5   70.8  665.3   0.8  17.0  72   6.7   0.0   0.0
13  6  5   sep  mon  90.9  126.5  686.5   7.0  21.3  42   2.2   0.0   0.0
14  6  5   sep  wed  92.9  133.3  699.[...]<jupyter_text>9. Applying Conditions on Dataframes Print all the columns and the rows where 'area' is greater than 0, 'wind' is greater than 1 and the 'temp' is greater than 15.<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
#df.head()
df_2 = df.loc[(df.area > 0) & (df.wind > 1) & (df.temp > 15), :]
print(df_2.head(20))<jupyter_output>X  Y month  day  FFMC    DMC     DC   ISI  temp  RH  wind  rain  area
138  9  9   jul  tue  85.8   48.3  313.4   3.9  18.0  42   2.7   0.0  0.36
139  1  4   sep  tue  91.0  129.5  692.6   7.0  21.7  38   2.2   0.0  0.43
140  2  5   sep  mon  90.9  126.5  686.5   7.0  21.9  39   1.8   0.0  0.47
141  1  2   aug  wed  95.5   99.9  513.3  13.2  23.3  31   4.5   0.0  0.55
142  8  6   aug  fri  90.1  108.0  529.8  12.5  21.2  51   8.9   0.0  0.61
143  1  2   jul  sat  90.0   51.3  296.3   8.7  16.6  53   5.4   0.0  0.71
144  2  5   aug  wed  95.5   99.9  513.3  13.2  23.8  32   5.4   0.0  0.77
145  6  5   aug  thu  95.2  131.7  578.8  10.4  27.4  22   4.0   0.0  0.90
147  8  3   sep  tue  84.4   73.4  671.9   3.2  24.2  28   3.6   0.0  0.96
148  2  2   aug  tue  94.8  108.3  647.1  17.0  17.4  43   6.7   0.0  1.07
149  8  6   sep  thu  93.7   80.9  685.2  17.9  23.7  25   4.5   0.0  1.12
150  6  5   jun  fri  92.5   56.4  433.3   7.1  23.2  39   5.4   0.0  1.19
151  9  9   jul  sun  90.[...]<jupyter_text>10. Dataframes Merge Perform an inner merge on two data frames df_1 and df_2 on  'unique_id' and print the combined dataframe.<jupyter_code>import pandas as pd
df_1 = pd.read_csv('https://query.data.world/s/vv3snq28bp0TJq2ggCdxGOghEQKPZo')
df_2 = pd.read_csv('https://query.data.world/s/9wVKjNT0yiRc3YbVJaiI8a6HGl2d74')
df_3 = pd.merge(df_1, df_2, how='inner', on='unique_id')
print(df_3.head(20))<jupyter_output>name                     address              city  \
0     arnie morton's of chicago   "435 s. la cienega blvd."     "los angeles"   
1                    art's deli       "12224 ventura blvd."     "studio city"   
2                 bel-air hotel      "701 stone canyon rd."         "bel air"   
3                    cafe bizou       "14016 ventura blvd."    "sher<NAME>"   
4                     campanile       "624 s. la brea ave."     "los angeles"   
5               chinois on main             "2709 main st."    "santa monica"   
6                        citrus         "6703 melrose ave."     "los angeles"   
7           fenix at the argyle         "8358 sunset blvd."    "w. hollywood"   
8                       granita       "23725 w. malibu rd."          "malibu"   
9                    grill  the           "9560 dayton way"   "beverly hills"   
10                        katsu       "1972 hillhurst ave."       "los feliz"   
11                  l'orange[...]<jupyter_text>11. Dataframe Append Append two datasets 'df_1' and 'df_2' and print the combined dataframe.<jupyter_code>import pandas as pd
df_1 = pd.read_csv('https://query.data.world/s/vv3snq28bp0TJq2ggCdxGOghEQKPZo')
df_2 = pd.read_csv('https://query.data.world/s/9wVKjNT0yiRc3YbVJaiI8a6HGl2d74')
print(df_1.head())
print(df_2.head())
df_3 = df_1.append(df_2)
print(df_3.head())<jupyter_output>name                     address             city  \
0  arnie morton's of chicago   "435 s. la cienega blvd."    "los angeles"   
1                 art's deli       "12224 ventura blvd."    "studio city"   
2              bel-air hotel      "701 stone canyon rd."        "bel air"   
3                 cafe bizou       "14016 ventura blvd."   "<NAME>"   
4                  campanile       "624 s. la brea ave."    "los angeles"   

            cuisine unique_id  
0     "steakhouses"       '0'  
1           "delis"       '1'  
2     "californian"       '2'  
3   "french bistro"       '3'  
4     "californian"       '4'  
                      name_2                  address_2           city_2  \
0  arnie morton's of chicago   "435 s. la cienega blv."    "los angeles"   
1         art's delicatessen      "12224 ventura blvd."    "studio city"   
2              hotel bel-air     "701 stone canyon rd."        "bel air"   
3                 cafe bizou      "14016 [...]<jupyter_text>12. Dataframe grouping Group the data 'df' by 'month' and 'day' and find the mean value for column 'rain' and 'wind'.<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
grp_by = df.groupby(['month', 'day'])
df_1 = grp_by[['rain', 'wind']].mean()
print(df_1.head(20))<jupyter_output>rain      wind
month day                    
apr   fri  0.000000  3.100000
      mon  0.000000  3.100000
      sat  0.000000  4.500000
      sun  0.000000  5.666667
      thu  0.000000  5.800000
      wed  0.000000  2.700000
aug   fri  0.066667  4.766667
      mon  0.000000  2.873333
      sat  0.000000  4.310345
      sun  0.025000  4.417500
      thu  0.000000  3.503846
      tue  0.300000  4.567857
      wed  0.000000  3.520000
dec   fri  0.000000  4.900000
      mon  0.000000  8.500000
      sun  0.000000  8.500000
      thu  0.000000  4.900000
      tue  0.000000  8.500000
      wed  0.000000  8.000000
feb   fri  0.000000  4.820000<jupyter_text>13. Creating New Column in a Dataframe Create a new column 'XY' which consist of values obtained from multiplying column 'X' and column 'Y'.<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
df['XY'] = df['X'] * df['Y']
print(df.head(20))<jupyter_output>X  Y month  day  FFMC    DMC     DC   ISI  temp  RH  wind  rain  area  XY
0   7  5   mar  fri  86.2   26.2   94.3   5.1   8.2  51   6.7   0.0   0.0  35
1   7  4   oct  tue  90.6   35.4  669.1   6.7  18.0  33   0.9   0.0   0.0  28
2   7  4   oct  sat  90.6   43.7  686.9   6.7  14.6  33   1.3   0.0   0.0  28
3   8  6   mar  fri  91.7   33.3   77.5   9.0   8.3  97   4.0   0.2   0.0  48
4   8  6   mar  sun  89.3   51.3  102.2   9.6  11.4  99   1.8   0.0   0.0  48
5   8  6   aug  sun  92.3   85.3  488.0  14.7  22.2  29   5.4   0.0   0.0  48
6   8  6   aug  mon  92.3   88.9  495.6   8.5  24.1  27   3.1   0.0   0.0  48
7   8  6   aug  mon  91.5  145.4  608.2  10.7   8.0  86   2.2   0.0   0.0  48
8   8  6   sep  tue  91.0  129.5  692.6   7.0  13.1  63   5.4   0.0   0.0  48
9   7  5   sep  sat  92.5   88.0  698.6   7.1  22.8  40   4.0   0.0   0.0  35
10  7  5   sep  sat  92.5   88.0  698.6   7.1  17.8  51   7.2   0.0   0.0  35
11  7  5   sep  sat  92.8   73.2  713.0  22.6  19.3  38   4.0   [...]<jupyter_text>14. Dataframe Pivot Table Group the data 'df' by 'month' and 'day' and find the mean value for column 'rain' and 'wind' using the pivot table command.<jupyter_code>import numpy as np
import pandas as pd
df = pd.read_csv('https://query.data.world/s/vBDCsoHCytUSLKkLvq851k2b8JOCkF')
df_1 = df.pivot_table(values = ['rain', 'wind'], index = ['month', 'day'])
print(df_1.head(20))<jupyter_output>rain      wind
month day                    
apr   fri  0.000000  3.100000
      mon  0.000000  3.100000
      sat  0.000000  4.500000
      sun  0.000000  5.666667
      thu  0.000000  5.800000
      wed  0.000000  2.700000
aug   fri  0.066667  4.766667
      mon  0.000000  2.873333
      sat  0.000000  4.310345
      sun  0.025000  4.417500
      thu  0.000000  3.503846
      tue  0.300000  4.567857
      wed  0.000000  3.520000
dec   fri  0.000000  4.900000
      mon  0.000000  8.500000
      sun  0.000000  8.500000
      thu  0.000000  4.900000
      tue  0.000000  8.500000
      wed  0.000000  8.000000
feb   fri  0.000000  4.820000<jupyter_text>15. Missing Values Print out the number of missing values in each column in the given dataframe.<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/Hfu_PsEuD1Z_yJHmGaxWTxvkz7W_b0')
print(df.isnull().sum())<jupyter_output>Ord_id                   0
Prod_id                  0
Ship_id                  0
Cust_id                  0
Sales                   20
Discount                55
Order_Quantity          55
Profit                  55
Shipping_Cost           55
Product_Base_Margin    109
dtype: int64<jupyter_text>16. Missing Values Percentage Find out the percentage of missing values in each column in the given dataset.<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/Hfu_PsEuD1Z_yJHmGaxWTxvkz7W_b0')
print(round(100*df.isnull().sum()/len(df.index), 2))#Round off percentage values to 2 decimial places.<jupyter_output>Ord_id                 0.00
Prod_id                0.00
Ship_id                0.00
Cust_id                0.00
Sales                  0.24
Discount               0.65
Order_Quantity         0.65
Profit                 0.65
Shipping_Cost          0.65
Product_Base_Margin    1.30
dtype: float64<jupyter_text>17. Removing Missing Values From the Rows Remove the missing values from the rows having greater than 5 missing values and then print the percentage of missing values in each column.<jupyter_code>import pandas as pd
df = pd.read_csv('https://query.data.world/s/Hfu_PsEuD1Z_yJHmGaxWTxvkz7W_b0')
df = df[df.isnull().sum(axis=1) <= 5]
print(round(100*df.isnull().sum()/len(df.index), 2))#Round off to 2 decimal places.<jupyter_output>Ord_id                 0.00
Prod_id                0.00
Ship_id                0.00
Cust_id                0.00
Sales                  0.00
Discount               0.42
Order_Quantity         0.42
Profit                 0.42
Shipping_Cost          0.42
Product_Base_Margin    1.06
dtype: float64<jupyter_text>18. Mean Imputation Impute the mean value at all the missing values of the column 'Product_Base_Margin' and then print the percentage of missing values in each column.<jupyter_code>import numpy as np
import pandas as pd
df = pd.read_csv('https://query.data.world/s/Hfu_PsEuD1Z_yJHmGaxWTxvkz7W_b0')
df.loc[np.isnan(df['Product_Base_Margin']), ['Product_Base_Margin']] = df['Product_Base_Margin'].mean()
print(round(100*df.isnull().sum()/len(df.index), 2))#Round off to 2 decimal places.<jupyter_output>Ord_id                 0.00
Prod_id                0.00
Ship_id                0.00
Cust_id                0.00
Sales                  0.24
Discount               0.65
Order_Quantity         0.65
Profit                 0.65
Shipping_Cost          0.65
Product_Base_Margin    0.00
dtype: float64
{'fraction_non_alphanumeric': 0.0975359896479215, 'fraction_numerical': 0.19922359411225535, 'mean_word_length': 1.62943011057556, 'pattern_counts': {'":': 0, '<': 58, '<?xml version=': 0, '>': 60, 'https://': 19, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '295114', 'license': 'MIT', 'n_tokens_mistral': 9694, 'n_tokens_neox': 8155, 'n_words': 2345, 'path': 'Practice/UpGrad Questions.ipynb', 'repo_name': 'theghostofc/Exploratory-Data-Analytics', 'size': '18555'}