<jupyter_start><jupyter_text>What is ```NumPy```? ```NumPy``` is the fundamental package for scientific computing in Python. It is a Python library that provides a multidimensional array object, various derived objects (such as masked arrays and matrices), and an assortment of routines for fast operations on arrays.If you're used to using ```MATLAB``` or using large numeric datasets, ```NumPy``` will be particularly useful for you. For example, I deal with lots of parameter sets and time series data while building my signaling models and I keep track of all of these using ```NumPy``` arrays.<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>At the core of the ```NumPy``` package, is the ```ndarray``` object. This encapsulates n-dimensional arrays of homogeneous data types  Creating an array<jupyter_code>a = [0,1,2,3]
x = np.array(a)
type(x)<jupyter_output><empty_output><jupyter_text>The basics  Python Lists vs. ```NumPy``` arrays - ```NumPy``` arrays have a **fixed size** at creation, unlike Python lists (which can grow dynamically). Changing the size of an ndarray will create a new array and delete the original.- The elements in a ```NumPy``` array are all required to be of the **same data type**, and thus will be the same size in memory. The exception: one can have arrays of (Python, including ```NumPy```) objects, thereby allowing for arrays of different sized elements. **Adding a scalar to a list**<jupyter_code>a = [1, 2, 3, 4]
4+a<jupyter_output><empty_output><jupyter_text>**Multiplying a scalar with a list**<jupyter_code>4*a<jupyter_output><empty_output><jupyter_text>**Adding a scalar to an array**<jupyter_code>x = np.array(a)
4+x<jupyter_output><empty_output><jupyter_text>**Multiplying a scalar with an array**<jupyter_code>4*x<jupyter_output><empty_output><jupyter_text>But why use NumPy? **Built in mathematical methods**```NumPy``` arrays facilitate advanced mathematical and other types of operations on large numbers of data. Typically, such operations are executed more efficiently and with less code than is possible using Python’s built-in sequences.**Popular in scientific community**A growing plethora of scientific and mathematical Python-based packages are using ```NumPy``` ```NumPy```; though these typically support Python-sequence input, they convert such input to ```NumPy``` arrays prior to processing, and they often output ```NumPy``` arrays. In other words, in order to efficiently use much (perhaps even most) of today’s scientific/mathematical Python-based software, just knowing how to use Python’s built-in sequence types is insufficient - one also needs to know how to use ```NumPy``` arrays.**Speed and Efficiency:**```Numpy``` arrays are much more compact than lists. Let's say you have data that's 100 x 100 x 100 = 1 million cells. That isn't terribly big, but it would take about 20 megabytes or so to store it in a list of lists. Whereas with ```NumPy```, it would take about 4MB (depending on the array's datatype). Your computer would be able to handle both of these data structures, but scalability is clearly an issue. **Timing basic numpy operations**<jupyter_code>import random 

matrix = []
for i in range(100):
    row = random.choices(range(100), k=10000)
    matrix.append(row)
numpy_matrix = np.random.randint(0, 100, (100, 10000))
%timeit means = [sum(row)/len(row) for row in matrix]
%timeit means = numpy_matrix.mean(1)<jupyter_output>7.56 ms ± 263 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
1.29 ms ± 150 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)<jupyter_text>Importing data using ```NumPy```<jupyter_code>f = "C:/Users/sksuzuki/Documents/EXCISION/lesson preps/data.csv"
np.loadtxt(f, delimiter=',')<jupyter_output><empty_output><jupyter_text>Indexing<jupyter_code>x = np.arange(15)
x<jupyter_output><empty_output><jupyter_text>**Slicing***One dimensional*<jupyter_code>x[2]
x[2:10]
x[::-1]<jupyter_output><empty_output><jupyter_text>*Multi dimensional*<jupyter_code>y = np.random.randint(15, size=15).reshape(3, 5)
y
y[2, 3]<jupyter_output><empty_output><jupyter_text>**Grabbing rows and columns**<jupyter_code># grab second row
y[1]
# grab second column
y[:, 1]
# grab second and third column
y[:, 1:3]
# grab last row
y[-1]<jupyter_output><empty_output><jupyter_text>**Grabbing windows of an array**<jupyter_code>y[:2, :2]<jupyter_output><empty_output><jupyter_text>**Grabbing positional information**<jupyter_code># Return the index of the array with the smallest value
y.argmin()<jupyter_output><empty_output><jupyter_text>Combining arrays<jupyter_code>x = np.array([1, 2, 3])
y = np.arange(15).reshape(3, 5)
print(y)
print(x)<jupyter_output>[[ 0  1  2  3  4]
 [ 5  6  7  8  9]
 [10 11 12 13 14]]
[1 2 3]<jupyter_text>**Multiplying arrays**<jupyter_code>4*x
x*y<jupyter_output><empty_output><jupyter_text>** Concatenate **<jupyter_code>np.concatenate((y, x), axis=0)<jupyter_output><empty_output><jupyter_text>**Transpose array**<jupyter_code>y = y.T
np.concatenate([y,x],axis=0)
print(y.shape)
print(x.shape)
x = np.array([[1, 2, 3]]) #Why?
print(x.shape)
x.squeeze().shape
np.concatenate([y, x], axis=0)<jupyter_output><empty_output><jupyter_text>Important ```NumPy``` Attributes and Methods **Reshaping an array**<jupyter_code>y = np.arange(15).reshape(3, 5)
print(y)<jupyter_output>[[ 0  1  2  3  4]
 [ 5  6  7  8  9]
 [10 11 12 13 14]]<jupyter_text>**Knowing the shape of your matrix**<jupyter_code>y.shape<jupyter_output><empty_output><jupyter_text>**Finding the size of your array**<jupyter_code>y.size<jupyter_output><empty_output><jupyter_text>**Sum along an axis**<jupyter_code>y.sum(axis=1)<jupyter_output><empty_output><jupyter_text>**Thresholding**<jupyter_code>(y > 5)
y[y>5]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12808052759458521, 'fraction_numerical': 0.027073932662270046, 'mean_word_length': 6.592885375494071, 'pattern_counts': {'":': 0, '<': 90, '<?xml version=': 0, '>': 92, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '46882', 'license': 'MIT', 'n_tokens_mistral': 2095, 'n_tokens_neox': 1952, 'n_words': 624, 'path': 'lectures/Intro to Numpy - Level 2.ipynb', 'repo_name': 'sksuzuki/How-to-Learn-to-Code', 'size': '5762'}