<jupyter_start><jupyter_text>NumPy this notebook is based on the SciPy NumPy tutorial Note that the traditional way to import numpy is to rename it np.  This saves on typing and makes your code a little more compact.<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>NumPy provides a multidimensional array class as well as a _large_ number of functions that operate on arrays.NumPy arrays allow you to write fast (optimized) code that works on arrays of data.  To do this, there are some restrictions on arrays:* all elements are of the same data type (e.g. float)* the size of the array is fixed in memory, and specified when you create the array (e.g., you cannot grow the array like you do with lists)The nice part is that arithmetic operations work on entire arrays&mdash;this means that you can avoid writing loops in python (which tend to be slow).  Instead the "looping" is done in the underlying compiled code  Array Creation and Properties There are a lot of ways to create arrays.  Let's look at a fewHere we create an array using `arange` and then change its shape to be 3 rows and 5 columns.  Note the _row-major ordering_&mdash;you'll see that the rows are together in the inner `[]` (more on this in a bit)<jupyter_code>a = np.arange(15)
a
a = np.arange(15).reshape(3,5)

print(a)<jupyter_output><empty_output><jupyter_text>an array is an object of the `ndarray` class, and it has methods that know how to work on the array data.  Here, `.reshape()` is one of the many methods. A NumPy array has a lot of meta-data associated with it describing its shape, datatype, etc.<jupyter_code>print(a.ndim)
print(a.shape)
print(a.size)
print(a.dtype)
print(a.itemsize)
print(type(a))
help(a)<jupyter_output><empty_output><jupyter_text>we can also create an array from a list<jupyter_code>b = np.array( [1.0, 2.0, 3.0, 4.0] )
print(b)
print(b.dtype)
print(type(b))<jupyter_output><empty_output><jupyter_text>we can create a multi-dimensional array of a specified size initialized all to 0 easily, using the `zeros()` function.<jupyter_code>b = np.zeros((10,8))
b<jupyter_output><empty_output><jupyter_text>There is also an analogous ones() and empty() array routine.  Note that here we can explicitly set the datatype for the array in this function if we wish. Unlike lists in python, all of the elements of a numpy array are of the same datatype<jupyter_code>c = np.eye(10, dtype=np.float64)
c<jupyter_output><empty_output><jupyter_text>`linspace` creates an array with evenly space numbers.  The `endpoint` optional argument specifies whether the upper range is in the array or not<jupyter_code>d = np.linspace(0, 1, 10, endpoint=False)
print(d)<jupyter_output><empty_output><jupyter_text>Quick Exercise:Analogous to `linspace()`, there is a `logspace()` function that creates an array with elements equally spaced in log.  Use `help(np.logspace)` to see the arguments, and create an array with 10 elements from $10^{-6}$ to $10^3$. we can also initialize an array based on a function<jupyter_code>f = np.fromfunction(lambda i, j: i + j, (3, 3), dtype=int)
f<jupyter_output><empty_output><jupyter_text>Array Operations most operations (`+`, `-`, `*`, `/`) will work on an entire array at once, element-by-element.Note that that the multiplication operator is not a matrix multiply (there is a new operator in python 3.5+, `@`, to do matrix multiplicaiton.Let's create a simply array to start with<jupyter_code>a = np.arange(12).reshape(3,4)
print(a)<jupyter_output><empty_output><jupyter_text>Multiplication by a scalar multiplies every element<jupyter_code>a*2<jupyter_output><empty_output><jupyter_text>adding two arrays adds element-by-element<jupyter_code>a + a<jupyter_output><empty_output><jupyter_text>multiplying two arrays multiplies element-by-element<jupyter_code>a*a<jupyter_output><empty_output><jupyter_text>Quick Exercise:What do you think `1./a` will do?  Try it and see We can think of our 2-d array as a 3 x 4 matrix (3 rows, 4 columns).  We can take the transpose to geta 4 x 3 matrix, and then we can do a matrix multiplication<jupyter_code>b = a.transpose()
b
a @ b<jupyter_output><empty_output><jupyter_text>We can sum the elements of an array:<jupyter_code>a.sum()<jupyter_output><empty_output><jupyter_text>Quick Exercise:`sum()` takes an optional argument, `axis=N`, where `N` is the axis to sum over.  Sum the elements of `a` across rows to create an array with just the sum along each column of `a`. We can also easily get the extrema<jupyter_code>print(a.min(), a.max())<jupyter_output><empty_output><jupyter_text>Universal functions universal functions work element-by-element.  Let's create a new array scaled by `pi`<jupyter_code>b = a*np.pi/12.0
print(b)
c = np.cos(b)
print(c)
d = b + c 
print(d)<jupyter_output><empty_output><jupyter_text>Quick Exercise:We will often want to write our own function that operates on an array and returns a new array.  We can do this just like we did with functions previously&mdash;the key is to use the methods from the `np` module to do any operations, since they work on, and return, arrays.Write a simple function that returns $\sin(2\pi x)$ for an input array `x`.  Then test it by passing in an array `x` that you create via `linspace()`  Slicing slicing works very similarly to how we saw with strings.  Remember, python uses 0-based indexing![](slicing.png)Let's create this array from the image:<jupyter_code>a = np.arange(9)
a<jupyter_output><empty_output><jupyter_text>Now look at accessing a single element vs. a range (using slicing) Giving a single (0-based) index just references a single value<jupyter_code>a[2]<jupyter_output><empty_output><jupyter_text>Giving a range uses the range of the edges to return the values<jupyter_code>print(a[2:3])
a[2:4]<jupyter_output><empty_output><jupyter_text>The `:` can be used to specify all of the elements in that dimension<jupyter_code>a[:]<jupyter_output><empty_output><jupyter_text>Multidimensional ArraysMultidimensional arrays are stored in a contiguous space in memory -- this means that the columns / rows need to be unraveled (flattened) so that it can be thought of as a single one-dimensional array.  Different programming languages do this via different conventions:![](row_column_major.png)Storage order:* Python/C use *row-major* storage: rows are stored one after the other* Fortran/matlab use *column-major* storage: columns are stored one after anotherThe ordering matters when * passing arrays between languages (we'll talk about this later this semester)* looping over arrays -- you want to access elements that are next to one-another in memory  * e.g, in Fortran:  ```  double precision :: A(M,N)  do j = 1, N     do i = 1, M        A(i,j) = …     enddo  enddo  ```    * in C  ```  double A[M][N];  for (i = 0; i < M; i++) {     for (j = 0; j < N; j++) {        A[i][j] = …     }  }    ```  In python, using NumPy, we'll try to avoid explicit loops over elements as much as possibleLet's look at multidimensional arrays:<jupyter_code>a = np.arange(15).reshape(3,5)
a<jupyter_output><empty_output><jupyter_text>Notice that the output of `a` shows the row-major storage.  The rows are grouped together in the inner `[...]`Giving a single index (0-based) for each dimension just references a single value in the array<jupyter_code>a[1,1]<jupyter_output><empty_output><jupyter_text>Doing slices will access a range of elements.  Think of the start and stop in the slice as referencing the left-edge of the slots in the array.<jupyter_code>a[0:2,0:2]<jupyter_output><empty_output><jupyter_text>Access a specific column<jupyter_code>a[:,1]<jupyter_output><empty_output><jupyter_text>Sometimes we want a one-dimensional view into the array -- here we see the memory layout (row-major) more explicitly<jupyter_code>a.flatten()<jupyter_output><empty_output><jupyter_text>we can also iterate -- this is done over the first axis (rows)<jupyter_code>for row in a:
    print(row)<jupyter_output><empty_output><jupyter_text>or element by element<jupyter_code>for e in a.flat:
    print(e)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09344059405940594, 'fraction_numerical': 0.010024752475247525, 'mean_word_length': 5.4648, 'pattern_counts': {'":': 0, '<': 111, '<?xml version=': 0, '>': 109, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '27', 'dir': 'jupyter-structured-clean-dedup', 'id': '12308', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2643, 'n_tokens_neox': 2546, 'n_words': 1003, 'path': 'content/03-python-numpy/numpy-basics.ipynb', 'repo_name': 'sbu-phy-ast-reu/reu-python-tutorial', 'size': '8080'}