<jupyter_start><jupyter_text>ANALYTICAL PYTHON  NUMPY Numpy is the most used library for fundamental scientific computing in python it provides multidimesinal array obejct.It is an extension module for Python,with a lot of it written in C.Since it is written in C this enables it to make the computation fast and ensures that the processing speeds are low.It offers a great computational speeds compared to build in python arrays. Dimensions in NumPy are called as Axes. Features of NumPy- NumPy arrays have their size fixed, unlike lists they cannot be changed- Elements in a NumPy array cannot be of differnt size, they only have to be of the same type.- We can store large numbers in a NumPy array, it consumes less size compared to the conventional arrays and offer much better effeciency.- It is an open source library.- Offers effecient calculations of mathematical functions that operate on arrays and matrices.  Installtion of NumPy Installation procedure for Windows,macOS and Linux Systems- For Begineers reccomended way to use NumPy is to first install Anaconda Installer from [Anaconda Website](www.anaconda.com)- choose a Python 3.6 graphical and download the file-  After the file gets downloaded click on the downloaded file to start the installation- Click Next- Read the licensing terms and click on I agree- Select an install for "Just Me unless you want the program for all users.- Select the destination where you want it to be installed.- Select Add Anaconda to my PATH environment variable which means that you would be able to use anaconda on the command prompt and click install.-  Click Finish  Alternatively NumPy can be installed with pip by using pip install numpy  Arrays in NumPy- Arrays is collection of same type of data types- Offers more features than the standard python library class array.array which can only handle 1 dimension and are indexed by non negetive integers.<jupyter_code>#To get the version of NumPy
print(np.__version__)
#Defining an array in NumPy
import numpy as np #importing numpy as np
arr=type(np.array([1,2,3,4]))
print(arr)
b=np.array([1,2,3,4,5,6,7,8,9,10])
new_arr=b.reshape(2,5)
print(new_arr)<jupyter_output>[[ 1  2  3  4  5]
 [ 6  7  8  9 10]]<jupyter_text>Dimensions of arrays - Dimensions of arrays are the number of keys or indices that we need to specify individual elements of the array- An array can be of multiple dimensions<jupyter_code>#Defining a 1 Dimension array using NumPy
print(np.array([1,2,3,4]))
#Defining a 2 Dimension array using NumPy
two_dimension_array=np.array([[1,2,3,4],[5,6,7,8]])
print(two_dimension_array)
#defining a 3 dimension array using NumPy
three_dimension_array=np.array([[1,2,3,4],[5,6,7,8],[9,10,11,12]])<jupyter_output>2<jupyter_text>Dimensions of an array can be checked by using shape function<jupyter_code>import numpy as np
dimensions=three_dimension_array.ndim
print(dimensions)<jupyter_output>2<jupyter_text>Total number of elements in an array can be checked by using size<jupyter_code>b=np.array([1,2,3,4,5,6,7,8,9,10])
new_arr=b.reshape(2,5)
print(new_arr.size)<jupyter_output>10<jupyter_text>Creation of arrays in numpy<jupyter_code>b=np.array([1,2,3,4,5,6,7,8,9,10])
print(b.dtype)<jupyter_output>int32<jupyter_text>Zeros,ones and  empty in numpy- Zeros is used to define a numpy array which has all the elements as zeros- Ones is used to define a numpy array which has all the elements as ones- Empty is used to created a numpy array with random values<jupyter_code>a=np.zeros((2,5))
print(a)
a=np.ones((2,6),dtype=np.int16)
print(a)
a=np.empty((2,5))
print(a)<jupyter_output>[[0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]]<jupyter_text>arange in NumPy- It is used to create sequence of arrays- It can be used to generate multidimension arrays<jupyter_code>#For 1 Dimension array
a=np.arange(10)
print(a)

#For 2 Dimension array
a= np.arange(10).reshape(2,5)
print(a)<jupyter_output>[0 1 2 3 4 5 6 7 8 9]
[[0 1 2 3 4]
 [5 6 7 8 9]]<jupyter_text>Mathematical operations in NumPy- Basic mathematical operations can be performed in NumPy element wise<jupyter_code>a=np.array([30,40,50,60])
b=np.array([5,6,4,1])

print(a+b)
print(a-b)
print(a*b)<jupyter_output>[35 46 54 61]
[25 34 46 59]
[150 240 200  60]<jupyter_text>Other operations that can be performed<jupyter_code>a=np.arange(15)
print(a.sum())
print(a.max())
print(a.min())<jupyter_output>105
14
0<jupyter_text>Matrix operations on NumPy- NumPy can perform matrix operations with high effeciency and less time  Dot multiplication of matrices<jupyter_code>a=np.array([30,40,50,60])
b=np.array([5,6,4,1])
dot_multiplication=np.dot(a,b)
print(dot_multiplication)<jupyter_output>650<jupyter_text>Square root of elements of Matrices<jupyter_code>print(a.T)
print(b.T)<jupyter_output>[5.47722558 6.32455532 7.07106781 7.74596669]
[2.23606798 2.44948974 2.         1.        ]<jupyter_text>Transpose of Matrix<jupyter_code>print(a.T)
print(b.T)<jupyter_output>[30 40 50 60]
[5 6 4 1]<jupyter_text>Pandas - Pandas is a fast, opensource tool used for data science and data analytics- It makes working with “relational” or “labeled” data more easy and intuative.- Pandas is used for reading and handling large amout of data.- It can handle missing data with ease.  Installation- Pandas can be installed using simple Pypi package when run on comand prompt > pip install pandas  Series- Series is a type of 1D array that can hold any type of data- Series can hold integer,string,float,etc. types of data  Defining a series<jupyter_code>information={
    "names":["Joseph","Daniel","Merry"],
    "class":[10,40,20]
}
student=pd.DataFrame(information)
print(student)
print(type(student))<jupyter_output>names  class
0    Jo     10
1  bala     40
2  ralo     20
<class 'pandas.core.frame.DataFrame'><jupyter_text>DataFrames- Dataframe is 2 Dimensional data structure that can store any type of data inside it.- DataFrame consists of 2 Dimensional Series dataset  Defining a DataFrame using dictionary<jupyter_code>information={
    "names":["Joesph","Robert","elly"],
    "class":[10,40,20]
}
student=pd.DataFrame(information)
print(student)
print(type(student))<jupyter_output>names  class
0  raju     10
1  bala     40
2  ralo     20
<class 'pandas.core.frame.DataFrame'><jupyter_text>Operations that can be performed on a DataFrame or Series- To calculate maximum value max() function can be used- To calculate minimum value min() function can be used- To show more infromation about the dataFrame describe() function can be used<jupyter_code>#For calculating maximum number
numbers = pd.Series([10, 5, 58])
print(numbers.max())

#For calculating minimum number

print(numbers.min())<jupyter_output>58
5<jupyter_text>- To get first 3 rows<jupyter_code>student.head(3)<jupyter_output><empty_output><jupyter_text>- To see the tehnical summary of the DataFrame<jupyter_code>student.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 3 entries, 0 to 2
Data columns (total 2 columns):
 #   Column  Non-Null Count  Dtype 
---  ------  --------------  ----- 
 0   names   3 non-null      object
 1   class   3 non-null      int64 
dtypes: int64(1), object(1)
memory usage: 176.0+ bytes<jupyter_text>Filter specific data from DataFrame- Data from a DataFrame can be easily filtered out for any specific condition  Finding students with age greater than<jupyter_code>print(student[student["age"]>10])<jupyter_output>names  age
1  bala   40
2  ralo   20<jupyter_text>To get number of rows and columns<jupyter_code>rows,columns= student.shape
print(rows,columns)<jupyter_output>3 2<jupyter_text>Converting DataFrame to different format- DataFrame can be converted into multiple formats such as csv,json,excel,etc- to_csv() is used to convert DataFrame into CSV fromat<jupyter_code>medals={
    "countries":["Germany","Australia","United States of America","China","Great Britain","France"],
    "medals":[32,36,77,70,48,25]
}

medals_data_1=pd.DataFrame(medals)
print(medals_data_1)
student.to_csv("updated_data.csv") #To convert file into CSV
student.to_excel("updated_data.xlsx") #To convert file into excel format<jupyter_output>countries  medals
0                   Germany      32
1                 Australia      36
2  United States of America      77
3                     China      70
4             Great Britain      48
5                    France      25<jupyter_text>More operations on DataFrame - Calculating Mean<jupyter_code>mean=medals_data_1["medals"].mean()
print(mean)<jupyter_output>48.0<jupyter_text>- Getting more information about the DataFrame<jupyter_code>des=medals_data_1[["countries","medals"]].describe
print(des)<jupyter_output><bound method NDFrame.describe of                   countries  medals
0                   Germany      32
1                 Australia      36
2  United States of America      77
3                     China      70
4             Great Britain      48
5                    France      25><jupyter_text>Combine data of multiple DataFrames<jupyter_code>new_df=pd.concat([student_data_1,student_data_2], join="outer")
print(new_df)<jupyter_output>names  class   age
0      raju   10.0   NaN
1      bala   40.0   NaN
2      ralo   20.0   NaN
0  Facebook    NaN  10.0
1    google    NaN  40.0
2     yahoo    NaN  20.0
3  linkedin    NaN  30.0
4   netflix    NaN  50.0
5     apple    NaN  35.0<jupyter_text>Reading tabular data file from Pandas- Pandas can be used to read tabular data.- Pandas read and convert the read tabular data into a DataFrame- If the data is not in the root directory in which we are running the file then we have to declare whole path of the file<jupyter_code>medal_info=pd.read_csv("Medals.csv")
print(medal_info)<jupyter_output><empty_output><jupyter_text>How Pandas is better than conventional methods- Convetional methods uses file handling to read data from a file- Pandas library makes it easier to read a file and perform suitable operations upon it<jupyter_code>tempratures=[]
with open("Medals.csv") as data_file:
    data=csv.reader(data_file)
    medal=[]
    for row in data:
            medal.append(int(row[1]))
print(medal)<jupyter_output><empty_output><jupyter_text>Checking Duplicate rows in a DataFrame<jupyter_code>information={
    "names":["raju","bala","ralo","raju","Elli","Lilly"],
    "age":[10,40,20,10,50,35]
}
student=pd.DataFrame.from_dict(information)
print(student.duplicated())<jupyter_output>0    False
1    False
2    False
3     True
4    False
5    False
dtype: bool<jupyter_text>Checking percentage change of Data- DataFrame.pct_change() is used to calculate change between the previous and the next elements<jupyter_code>change_in_marks=pd.DataFrame({
    '2011':[98, 88,77],
    '2012':[92, 98,89], 
    '2013':[77,87,90]},
    index = ['Jacob','Joseph','Angela'])

print(change_in_marks.pct_change(axis='columns'))<jupyter_output>2011      2012      2013
Jacob    NaN -0.061224 -0.163043
Joseph   NaN  0.113636 -0.112245
Angela   NaN  0.155844  0.011236<jupyter_text>Removing NaN values from DataFrame- NaN values are empty values in the dataset- NaN values can be converted into zeros in order to avoid errors usng the function fillna(0)<jupyter_code>change_in_marks=pd.DataFrame({
    '2011':[98, 88,77],
    '2012':[92, 98,89], 
    '2013':[77,87,90]},
    index = ['Jacob','Joseph','Angela'])

print(change_in_marks.pct_change(axis='columns').fillna(0))<jupyter_output>2011      2012      2013
Jacob    0.0 -0.061224 -0.163043
Joseph   0.0  0.113636 -0.112245
Angela   0.0  0.155844  0.011236
{'fraction_non_alphanumeric': 0.09554528811786243, 'fraction_numerical': 0.054572399965129456, 'mean_word_length': 4.48112756808409, 'pattern_counts': {'":': 8, '<': 92, '<?xml version=': 0, '>': 94, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '28', 'dir': 'jupyter-structured-clean-dedup', 'id': '194424', 'license': 'MIT', 'n_tokens_mistral': 4180, 'n_tokens_neox': 3766, 'n_words': 1260, 'path': 'Sumit_Gaur_Numpy and Pandas.ipynb', 'repo_name': 'Sumit3301/Open-contributions', 'size': '11471'}