<jupyter_start><jupyter_text>Agenda  1.  [Review](0)2.  [Numpy continued](2)2.  [Exercise](10) 3.  [Exercise](12)      Review Each array has attributes ``ndim`` (the number of dimensions), ``shape`` (the size of each dimension), and ``size`` (the total size of the array):<jupyter_code>import numpy as np
x3 = np.random.randint(10, size=(3, 4, 5))  # Three-dimensional array
print("x3 ndim: ", x3.ndim)
print("x3 shape:", x3.shape)
print("x3 size: ", x3.size)<jupyter_output>x3 ndim:  3
x3 shape: (3, 4, 5)
x3 size:  60<jupyter_text>Array Arithmetic Vectorized operations in NumPy are implemented via *ufuncs*, whose main purpose is to quickly execute repeated operations on values in NumPy arrays. We also call this element-wise arithmetic.<jupyter_code>x = np.array([1,2,3,4])
y = np.array([2,3,4,5])
print(x+y)
np.arange(5)/np.arange(1,6)
print(np.arange(5)/np.arange(1,6))
print(np.arange(5))
print(np.arange(1,6))

np.add(x, 2)<jupyter_output><empty_output><jupyter_text>All the arithmetic operators implemented in NumPy:| Operator	    | Equivalent ufunc    | Description                           ||---------------|---------------------|---------------------------------------||``+``          |``np.add``           |Addition (e.g., ``1 + 1 = 2``)         ||``-``          |``np.subtract``      |Subtraction (e.g., ``3 - 2 = 1``)      ||``-``          |``np.negative``      |Unary negation (e.g., ``-2``)          ||``*``          |``np.multiply``      |Multiplication (e.g., ``2 * 3 = 6``)   ||``/``          |``np.divide``        |Division (e.g., ``3 / 2 = 1.5``)       ||``//``         |``np.floor_divide``  |Floor division (e.g., ``3 // 2 = 1``)  ||``**``         |``np.power``         |Exponentiation (e.g., ``2 ** 3 = 8``)  ||``%``          |``np.mod``           |Modulus/remainder (e.g., ``9 % 4 = 1``)|  Array Indexing: Accessing Single Elements Done the exact same way as with Python lists. In multi-dimensional numpy arrays, we can index multiple times to access values.<jupyter_code>x1 = np.random.randint(10, size=6)  # One-dimensional array
x1[0]
x1[-1]
print(x1)
x1[2:6:2]


import numpy as np
x2 = np.random.randint(10, size=(3, 4))  # Two-dimensional array
print(x2)
#x2[0, 0]
#x2[1, 0]
x2[::-1,::-1]
#np.flip(x2,axis=1)
#x2[1, -1]<jupyter_output>[[3 0 3 1]
 [2 6 4 6]
 [9 4 1 3]]<jupyter_text>Slicing Arrays To access subsections of a larger array, we use the : operation to *slice* a specified part of the array. ``` pythonx[start:stop:step]```If any of these are unspecified, they default to the values ``start=0``, ``stop=``*``size of dimension``*, ``step=1``.We'll take a look at accessing sub-arrays in one dimension and in multiple dimensions.<jupyter_code>x = np.arange(10)
x[:3]  # first three elements
x[5:] #elements after index 5 
x[4:7]  # middle sub-array
x[::-1]  # all elements, reversed<jupyter_output><empty_output><jupyter_text>Reshaping ArraysSimply call array.reshape(new_shape) on your old array with a new shape. Make sure the shape size is compatible with the array length (factors).<jupyter_code>grid = np.arange(1, 11).reshape((5, 2))
print(grid)<jupyter_output>[[ 1  2]
 [ 3  4]
 [ 5  6]
 [ 7  8]
 [ 9 10]]<jupyter_text>ExerciseCreate a numpy array with 2 dimensions, a shape of 5x5, and calculate the array's size. Solution<jupyter_code><jupyter_output><empty_output><jupyter_text>Exercise Create a 1-D array of 8 values using np.arange(8), and then reshape it to a 3-dimensional array using the array.reshape(new_shape) function. *HINT* look at the cubic of 8 to find the appropriate shape. Solution<jupyter_code>n1 = np.arange(8)
print(n1)
n2 = n1.reshape((2,2,2))
print(n2)<jupyter_output>[0 1 2 3 4 5 6 7]
[[[0 1]
  [2 3]]

 [[4 5]
  [6 7]]]<jupyter_text>Exercise<jupyter_code><jupyter_output><empty_output><jupyter_text>Numpy continuedWe will explore Numpy ufuncs, which stand for 'Universal Functions', thatoperate on the ndarray object.ufuncs are used to implement vectorization in NumPy which is way faster than iterating over elements.They also provide broadcasting and additional methods like reduce, accumulate etc. that are very helpful for computation.ufuncs also take additional arguments, like:- where - boolean array or condition defining where the operations should take place.- dtype - defining the return type of elements.- out - output array where the return value should be copied. Adding 2 python lists togetherx = [3, 5, 7, 9]y = [4, 6, 8, 10]z = np.add(x, y)print(z)  print(type(z))NOTE: The shape of the numpy objects must be the same.<jupyter_code>x = np.array([3, 5, 7, 9])
y = np.array([4, 6, 8, 10])
z = x+y
print(z)<jupyter_output>[ 7 11 15 19]<jupyter_text>Subtractionx = [13, 25, -7, 39]y = [4, 6, 8, 10]z = np.subtract(x, y)print(z)print(type(z))<jupyter_code>z = np.subtract(x, y)<jupyter_output><empty_output><jupyter_text>Multiplicationx = [13, 25, -7, 39]y = [4, 6, 8, 10]z = np.multiply(x, y)print(z)print(type(z))<jupyter_code>x = [13, 25, -7, 39]
y = [4, 6, 8, 10]
z = np.multiply(x, y)
print(z)<jupyter_output>[ 52 150 -56 390]<jupyter_text>Divisionimport numpy as nparr1 = np.array([10, 20, 30, 40, 50, 60])arr2 = np.array([3, 5, 8, 13, 21, 34])newarr = np.divide(arr1, arr2)print(newarr) Powersarr1 = np.array([2, 3, 4, 5, 6, 7])arr2 = np.array([7, 6, 5, 4, 3, 2])newarr = np.power(arr1, arr2)print(newarr) Reminder/ Modulusarr1 = np.array([10, 20, 30, 40, 50, 60])arr2 = np.array([3, 7, 6, 14, 8, 13])newarr = np.remainder(arr1, arr2)print(newarr) arr1 = np.array([10, 20, 30, 40, 50, 60])arr2 = np.array([3, 7, 9, 8, 2, 33])newarr = np.mod(arr1, arr2)print(newarr) Quotient and Modarr1 = np.array([10, 20, 30, 40, 50, 60])arr2 = np.array([2, 5, 9, 8, 13, 43])newarr = np.divmod(arr1, arr2)    Returns 2 arrays !!!print(newarr) Absolute Valuearr = np.array([-1, -2, 10, 2, 13, -9])newarr = np.absolute(arr)print(newarr)<jupyter_code>arr1 = np.array([2, 3, 4, 5, 6, 7]) 
arr2 = np.array([7, 6, 5, 4, 3, 2])
print(np.power(arr1,arr2))<jupyter_output>[ 128  729 1024  625  216   49]<jupyter_text>Try them Out!<jupyter_code><jupyter_output><empty_output><jupyter_text>Exercise a1 = np.array([[1,2],               [3,4],               [5,6]])a2 = np.array([[7,8],               [9,10],               [10,11]])Reshape the two arrays into 1-D arrays and add them, print the sum.<jupyter_code>a1 = np.array([[1,2], [3,4], [5,6]])
a2 = np.array([[7,8], [9,10], [10,11]])
a1 = a1.reshape(6,)
a2 = a2.reshape(6,)
print(np.add(a1,a2))<jupyter_output>[ 8 10 12 14 15 17]<jupyter_text>Numpy ufunc - Rounding DecimalsThere are primarily five ways of rounding off decimals in NumPy:- truncation - remove the decimal portion of a numberimport numpy as nparr = np.trunc([-3.1666, 3.6667, 111.23334])print(arr)    - rounding - equivalent to python roundingarr = np.around(3.1417, 2)print(arr)     - floor - Returns the next lower integer numberarr = np.floor([-3.1666, 3.6667])print(arr)     - ceil - Returns the next higher integer numberarr = np.ceil([-3.1666, 3.6667])print(arr)      Exercise: round pi to the nearest 5 digits using numpy and use to calculate the area of a circle with radius 3.<jupyter_code>print(np.pi)<jupyter_output>3.141592653589793<jupyter_text>Summing all elementsWe can use the Numpy sum function to add up all of the values in1 or more np.arraysimport numpy as nparr1 = np.array([3, 4, 1])arr2 = np.array([-1,-2, 7])newarr = np.sum([arr1, arr2])print(newarr)<jupyter_code>import numpy as np

arr1 = np.array([3, 4, 1])
arr2 = np.array([-1,-2, 7])

newarr = np.sum([arr1, arr2])
print(newarr) 
big_array = np.random.rand(1000000)

%timeit min(big_array)
%timeit np.min(big_array)
#np.max<jupyter_output>10 loops, best of 5: 104 ms per loop
The slowest run took 5.56 times longer than the fastest. This could mean that an intermediate result is being cached.
1000 loops, best of 5: 413 µs per loop<jupyter_text>Exercise:There are 10 customers with account holdings given below. We want to separate customers with an total holding of more than 1,000 dollars (commercial accounts) and those with less than 1,000 dollars (personal accounts). 1) what is the total commercial account value?2) using np.max() what is the highest account holding amongst the personal accounts? We want to aggregate all the personal and commercial account holdings into separate arrays. 3) Find the mean and stdev of the personal account list. 4) Find the median and mean of the commercial account list.   We'd like to plot the commercial account holdings against the personal account holdings. 5) Import matplotlib and use the plt.scatter(x,y) method to print a graph depicting this relationship. c1 = np.array([155, 204, 312, 56])c2 = np.array([233, 245, 333])c3 = np.array([802, 7, 382, 9329, 58392])c4 = np.array([222, 453, 10235, 9929])c5 = np.array([902, 75, 29394])c6 = np.array([65, 45])c7 = np.array([550, 300])c8 = np.array([200, 120])c9 = np.array([550, 1200])c10 = np.array([5312, 0])<jupyter_code>import numpy as np
c1 = np.array([155, 204, 312, 56]) 
c2 = np.array([233, 245, 333]) 
c3 = np.array([802, 7, 382, 9329, 58392]) 
c4 = np.array([222, 453, 10235, 9929]) 
c5 = np.array([902, 75, 29394]) 
c6 = np.array([65, 45]) 
c7 = np.array([550, 300]) 
c8 = np.array([200, 120]) 
c9 = np.array([550, 1200]) 
c10 = np.array([5312, 0])
c11 = np.array([150, 200, 20])
clist = [c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11]
comm = np.array(0)
pers = np.array(0)
for i in clist:
  if i.sum() > 1000:
    comm = np.append(comm, i) 
    #np.add; np.insert, np.concatenate
  else:
    pers = np.append(pers,i)
comm = np.delete(comm, 0)
pers = np.delete(pers, 0)
print('commercial value sum: $', np.sum(comm))
print('highest personal holding: $', np.max(pers))
print(np.mean(pers))
print(np.std(pers))
print(np.median(comm))
print(np.mean(comm))
print(len(comm), len(pers))
print(pers)
plt.scatter(pers, comm)
plt.show()<jupyter_output><empty_output><jupyter_text>This notebook is part of a course at www.codeimmersives.com called Data Science. If you accessed this notebook outside the course, you can get more information about this course online by clicking here.   Intro Dataloading Example: What is the Average Height of US Presidents? Aggregates available in NumPy can be extremely useful for summarizing a set of values.As a simple example, let's consider the heights of all US presidents.This data is available in the file *president_heights.csv*, which is a simple comma-separated list of labels and values:<jupyter_code>!head -4 data/president_heights.csv<jupyter_output>head: cannot open 'data/president_heights.csv' for reading: No such file or directory<jupyter_text>We'll use the Pandas package, which we'll explore more fully next class to read the file and extract this information (note that the heights are measured in centimeters).<jupyter_code>import pandas as pd
import numpy as np
data = pd.read_csv('president_heights.csv')
heights = np.array(data['height(cm)'])
print(heights)<jupyter_output>[189 170 189 163 183 171 185 168 173 183 173 173 175 178 183 193 178 173
 174 183 183 168 170 178 182 180 183 178 182 188 175 179 183 193 182 183
 177 185 188 188 182 185]<jupyter_text>Now that we have this data array, compute summary statistics (mean, stdev, min, and max of height):<jupyter_code>print("Mean height:       ", heights.mean())
print("Standard deviation:", heights.std())
print("Minimum height:    ", heights.min())
print("Maximum height:    ", heights.max())<jupyter_output>Mean height:        179.73809523809524
Standard deviation: 6.931843442745892
Minimum height:     163
Maximum height:     193<jupyter_text>Now calcualte the quartiles using np.percentile(data, percentile) for the 1st quartile (25%), 2nd quartile (50%), and third quartile (75%).<jupyter_code>print("25th percentile:   ", np.percentile(heights, 25))
print("Median:            ", np.median(heights))
print("75th percentile:   ", np.percentile(heights, 75))<jupyter_output>25th percentile:    174.25
Median:             182.0
75th percentile:    183.0<jupyter_text>Once you get the average height, try plotting this in a histogram chart (hint: use plt.hist())<jupyter_code>import matplotlib.pyplot as plt
plt.hist(heights)
plt.title('Height Distribution of US Presidents')
plt.xlabel('height (cm)')
plt.ylabel('number');<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15252434334342524, 'fraction_numerical': 0.08256280173471893, 'mean_word_length': 4.330135194068905, 'pattern_counts': {'":': 0, '<': 75, '<?xml version=': 0, '>': 76, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '273130', 'license': 'BSD-Source-Code', 'n_tokens_mistral': 5132, 'n_tokens_neox': 4358, 'n_words': 1384, 'path': 'CI_Data_Science_Lesson_07_NO_ANSWERS.ipynb', 'repo_name': 'MaxDGU/datasciencenotebooks', 'size': '12221'}