<jupyter_start><jupyter_text>Introduction to NumPy `numpy` is at the heart of the Scientific Python Ecosystem. It provides multi-dimensional array manipulation, elementary mathematical functions, random number generators, linear algebra routines, ...`numpy` is a very large package with several libraries (for all of those things mentioned above). **DO NOT** try to remember all the functions and methods in one sitting. Prioritize understanding some specific concepts we will highlight, which will be much more useful. Leave the listing of functions to the `numpy` documentation. After all, it's free real estate. [<NAME>as PyCon 2017 Keynote](https://speakerdeck.com/jakevdp/the-unexpected-effectiveness-of-python-in-science?slide=52) This notebook was adapted from an older (less extensive) version at https://github.com/krittikaiitb/tutorials. If you are interested in using Python for Astronomy, do check out the repository and suggest additions! External Resources1. A very good introduction to Python from a Data Science PoV is the [Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/index.html) by [<NAME>](https://jakevdp.github.io/). The chapter on NumPy is [Introduction to NumPy](https://jakevdp.github.io/PythonDataScienceHandbook/02.00-introduction-to-numpy.html)2. [NumPy documentation](https://numpy.org/doc/stable/) and [NumPy reference](https://numpy.org/doc/stable/reference/index.html): Don't read the entire thing in one sitting!  Overview:1. **NumPy Basics**   We will go a bit fast here. I will highlight important points and things to remember. (30 min)2. **Multi-dimensional NumPy arrays**   We will take it a bit slow here. Key takeaways: slicing, `axis` keyword. (30 min)3. **NumPy libraries**   Mixed bag of NumPy functions. (20 min)<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>The above line imports the package numpy (which includes all the functions we will be using). The part `as np` renames it to `np` (just for this program, don't worry), which is slightly more convenient.   Creating a `numpy` array There are a lot of ways to create an array (and we will see some in this notebook). The simplest is to create an array from an array-like object (lists, tuples, sets)<jupyter_code>array_1 = np.array([1, 2, 3, 4])
array_2 = np.asarray([1, 2, 3, 4])
array_1_copy = np.array(array_1)
array_2_ref = np.asarray(array_2)
array_1_copy[0] = 100
array_2_ref[0] = 100
print(array_1)
print(array_2)<jupyter_output>[1 2 3 4]
[100   2   3   4]<jupyter_text>Element-wise operations We can do the usual arithmetic operations on a `numpy` array element wise. For example, multiplication by a scalar:<jupyter_code>x = np.array([1, 2, 3, 4])
# x = [1, 2, 3, 4]
# x = (1, 2, 3, 4)
y = x * 2


print(f"For {type(x)}:\nx = {x}\ny = {y}")<jupyter_output>For <class 'numpy.ndarray'>:
x = [1 2 3 4]
y = [2 4 6 8]<jupyter_text>Element wise operations are great. But they can be done with lists as well (using `for` loops or a list comprehension). What makes `numpy` special?<jupyter_code>list_ = list(range(10_000))
array_ = np.array(list_)
%timeit y = [l*2 for l in list_]
%timeit y = array_ * 2<jupyter_output>3.78 µs ± 35.9 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)<jupyter_text>The values above will be different for your computer, but it is clearly obvious that `numpy` arrays are faster. `numpy` also supports indexing with lists, instead of only integers or slices, whereas lists don't.<jupyter_code># Here's another way to create a numpy array
x = np.arange(0, 100, 1)
indices = [27, 7, 17, 1]
x[indices]<jupyter_output><empty_output><jupyter_text>Trying a similar trick with a list instead: (this should give an error!)<jupyter_code>x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
print(x[indices])<jupyter_output><empty_output><jupyter_text>Similar to the `len` function, numpy also has several functions to describe an array. Several are also available as methods, which can be used as follows:<jupyter_code>x = np.array([[1, 2, 3, 4],
              [5, 6, 7, 8],
              [9, 10, 11, 12]])
# print(x)<jupyter_output><empty_output><jupyter_text>Note:  2D arrays like above require the same number of elements in each row. If you remove say 6 from the above, you'll get a `Warning` from `numpy` talking about "ragged nested sequences". It is unlikely you will encounter a need for these much (unless you want to analyze data from particle accelerators, for example) so we will ignore them for now.Note:  If you want to analyze data from particle accelerators, go watch this video: https://www.youtube.com/watch?v=2NxWpU7NArk<jupyter_code>print(x.dtype)  # data-type. note the difference between int64 and Python's int
print(x.ndim)  # number of dimensions. This is a 2D array
print(x.shape)  # Shape of array
print(x.size)  # Total number of elements in array (product of elements in shape)<jupyter_output>int64
2
(3, 4)
12<jupyter_text>Note:  `numpy` arrays have a method called `astype`<jupyter_code>x.dtype
y = x.astype("float")
y.dtype<jupyter_output><empty_output><jupyter_text>Note:  `numpy` arrays have a method called `reshape`<jupyter_code>x.shape
y = x.reshape((2, 6))
y<jupyter_output><empty_output><jupyter_text>Multi-dimensional `numpy` arrays can be indexed using multiple indices<jupyter_code>x
x[1, 2]<jupyter_output><empty_output><jupyter_text>Element wise operations also work between arrays of the same shape<jupyter_code>x*x<jupyter_output><empty_output><jupyter_text>If you want proper matrix multiplication, convert the `numpy` arrays to a `matrix`, or use `numpy.dot`<jupyter_code>np.matrix(x) * np.matrix(x.T)
np.dot(x, x.T)  # x.dot(x)<jupyter_output><empty_output><jupyter_text>Relational operators are also operate element-wise on the array:<jupyter_code>x > 5
x = np.arange(10)
(x < 6)
x > 3<jupyter_output><empty_output><jupyter_text>The array of `dtype` `bool` can be used to index an array (it works like you would expect)<jupyter_code>y = np.copy(x) # Simply typing y = x doesnot make a new array, it just adds a reference to the old one.
               # Changes in y will then be reflected in x
y[y > 5] = 0
y
x = np.arange(10)
y = x[x**2 - 3*x - 5 >= 0]
y<jupyter_output><empty_output><jupyter_text>Wondering about logical operators like AND, OR and NOT. You can simply use the bitwise operators `&`, `|` and `~`<jupyter_code>x[(x < 6) & (x > 3)]
x[(x < 2) | (x > 7)]
x[~(x == 5)]<jupyter_output><empty_output><jupyter_text>Break: 5 minTry out the cells, change up values and see what happens. If you have questions, put them up on slido for our mods to answer. We can have a QnA after the content is over on Teams if they are not resolved by then.  Summary:1. Create `numpy` arrays from sequences using `np.array`2. Operations work element-wise: array-scalar and array-array of same shape3. Arithmetic operations, relational operations and bit-wise logical operators all work!4. Array attributes: `dtype`, `ndim`, `shape`, `size`5. So far, you can index a numpy array with:    1. An integer    2. A slice    3. A list of integers    4. Another numpy array of the same `shape` with `dtype` `bool` If we want the indices where a certain condition is satisified, we can use `numpy.where`.<jupyter_code>x = np.array([0, 1, 2, 3, 4, 5, 6, 7, 0.5, 0.6])
indices = np.where(x < 5)
indices<jupyter_output><empty_output><jupyter_text>`indices` is not a `numpy` array!It is a `tuple` with as many arrays as the dimension of the array `x`.<jupyter_code>x[indices]
x = np.array([
    [1, 2, 3, 4],
    [5, 6, 7, 8],
    [9, 10, 11, 12]])
indices = np.where(x > 5)
indices
x[indices]<jupyter_output><empty_output><jupyter_text>Have I been keeping any other way of indexing from you? SlicingTinker around with the next cell<jupyter_code>x
x[:, :]
ind1 = [True, False, True]
ind2 = [True, False, True, True]
x[ind1, ind2]<jupyter_output><empty_output><jupyter_text>We can also delete elements using `numpy.delete`.<jupyter_code>x
indices = np.where(x[:, 0] > 1)
indices
y = np.delete(x, indices, axis=1)
print(y)
x.shape<jupyter_output><empty_output><jupyter_text>Note the argument of the function called `axis`. For a 2D array, `axis=0` is along the "rows", and `axis=1` is along the "columns".<jupyter_code>x
np.sum(x)<jupyter_output><empty_output><jupyter_text>Break: 5 minTry out the above cells, change up values and see what happens. If you have questions, put them up on slido for our mods to answer. We can have a QnA after the content is over on Teams if they are not resolved by then.In particular, make sure you understand the parts about slicing a multi-dimensional array, how `numpy.where` works and the `axis` keyword.  Summary:1. Create `numpy` arrays using `np.array`2. Operations work element-wise: array-scalar and array-array of same shape3. Array attributes: `dtype`, `ndim`, `shape`, `size`4. So far, you can index a numpy array with:    1. An integer    2. A slice    3. A list of integers    4. Another numpy array of the same `shape` with `dtype` `bool`    5. A tuple of numpy arrays (like in `numpy.where`) although this always return a 'flattened' array    6. A slice, for each dimension, separated by a comma  Broadcasting Broadcasting is magic. We will just have a look at an example here.To know more about the rules for broadcasting (they are simple, but will take some time to explain), and more examples and practical applications, refer the relevant chapter in [Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/02.05-computation-on-arrays-broadcasting.html).<jupyter_code>x = np.arange(3)
y = np.ones((3, 3))
x
y
y + x.reshape(-1, 1)<jupyter_output><empty_output><jupyter_text>NumPy Speedrun  Numerical Ranges Other useful functions in numpy like `numpy.arange`, `numpy.linspace` often come in handy. We have already seen `arange`<jupyter_code>np.arange(0, 1, 0.1)
np.linspace(0, 1, 5)
np.geomspace(1, 16, 5)
np.logspace(0, 5, 6)<jupyter_output><empty_output><jupyter_text>Mathematical Functions and Constants<jupyter_code>np.pi, np.e
x = np.arange(-np.pi, np.pi, 0.1)
y = np.sin(x)
x_abs = np.abs(x)
y2 = np.exp(x)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11021133715188623, 'fraction_numerical': 0.025380209362038317, 'mean_word_length': 4.932630345635618, 'pattern_counts': {'":': 0, '<': 100, '<?xml version=': 0, '>': 104, 'https://': 9, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '24', 'dir': 'jupyter-structured-clean-dedup', 'id': '272424', 'license': 'MIT', 'n_tokens_mistral': 3568, 'n_tokens_neox': 3347, 'n_words': 1284, 'path': 'tutorial.ipynb', 'repo_name': 'vedshenoy/numpy_tutorial', 'size': '10142'}