<jupyter_start><jupyter_text>NumPy - Numeric python  NumPy (np) is the premier Python package for scientific computinghttps://numpy.orgIts powerful comes from the N-dimensional array objectnp is a *lower*-level numerical computing library. This means that, while you can use it directly, most of its power comes from the packages built on top of np:* Pandas (*Pan*els *Da*tas)* Scikit-learn (machine learning)* Scikit-image (image processing)* OpenCV (computer vision)* more... Importing NumPyConvention: use np alias<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>NumPy basics Arrays are designed to:* handle vectorized operations lists are not    * if you apply a function it is performed on every item in the array, rather than on the whole array object* store multiple items of the same data type* have 0-based indexing* Missing values can be represented using `np.nan` object    * the object `np.inf` represents infinite* Array size cannot be changed, should create a new array* An equivalent numpy array occupies much less space than a python list of lists Create Arrayhttps://docs.scipy.org/doc/numpy-1.13.0/user/basics.creation.html<jupyter_code># Build array from Python list
vector = np.array([1,2,3])
vector
# matrix with zeros 
np.zeros(3,4)
# matrix with 1s
np.ones((3,4), dtype=int)
# matrix with a constant value
np.full((3,4), 10)
# Create a 4x4 identity matrix
np.eye(4)        
# arange - numpy range
np.arange(10, 30, 2)
# evenly spaced numbers over a specified interval
np.linspace(1, 10, 20)<jupyter_output><empty_output><jupyter_text>Random datahttps://docs.scipy.org/doc/numpy-1.13.0/reference/routines.random.html<jupyter_code># Create an array filled with random values
np.random.random((3,4))        
# Create an array filled with random values from the standard normal distribution
np.random.randn((3,4))    
# Generate the same random numbers every time
# Set seed
np.random.seed(100)<jupyter_output><empty_output><jupyter_text>```python Create the random staters = np.random.RandomState(100)``` Basic array attributes:* shape: array dimension* size: Number of elements in array* ndim: Number of array dimension (len(arr.size))* dtype: Data-type of the array* T: The transpose of the array<jupyter_code>matrix = np.array([[1,2,3],[4,5,6],[7,8,9],[10,11,12]])
matrix
# let's check them out 
matrix.shape<jupyter_output><empty_output><jupyter_text>Reshaping<jupyter_code>matrix
# Reshaping
matrix_reshaped = matrix.reshape(2,6)
matrix_reshaped<jupyter_output><empty_output><jupyter_text>Slicing/Indexing<jupyter_code># List-like
matrix_reshaped[1][1]
matrix_reshaped[1,3]
matrix_reshaped[1,:3]
# iterrating ... let's print the elements of matrix_reshaped


# Fun arrays
checkers_board = np.zeros((8,8),dtype=int)
checkers_board[1::2,::2] = 1
checkers_board[::2,1::2] = 1
print(checkers_board)<jupyter_output><empty_output><jupyter_text>Create a 2d array with 1 on the border and 0 inside Performance test_list = list(range(int(1e6)))test_vector = np.array(test_list)<jupyter_code>%%timeit
sum(test_list)
%%timeit
np.sum(test_vector)<jupyter_output><empty_output><jupyter_text>https://numpy.org/devdocs/user/quickstart.htmluniversal-functions Matrix operationshttps://www.tutorialspoint.com/matrix-manipulation-in-pythonArithmetic operators on arrays apply elementwise.  A new array is created and filled with the result.Array broadcastinghttps://docs.scipy.org/doc/numpy/user/basics.broadcasting.htmlThe term broadcasting describes how numpy treats arrays with different shapes during arithmetic operations. Subject to certain constraints, the smaller array is “broadcast” across the larger array so that they have compatible shapes.  https://www.tutorialspoint.com/numpy/numpy_broadcasting.htm<jupyter_code>matrix
matrix + np.array([1,2,3,4]).reshape(4,1)
matrix * np.array([1,2,3,4]).reshape(4,1)
matrix2 = np.array([[1,2,3],[5,6,7],[1,1,1],[2,2,2]])
matrix * matrix2
# matrix multiplication
matrix.dot(np.array([1,2,3]).reshape(1,3))
# matrix multiplication - more recently
matrix@(np.array([1,2,3]).reshape(3,1))
# stacking arrays together
np.vstack((matrix,matrix2))
np.hstack((matrix,matrix2))
# splitting arrays 
np.vsplit(matrix,2)
np.hsplit(matrix,(2,3))<jupyter_output><empty_output><jupyter_text>Copy<jupyter_code>matrix
# shallow copy - looks at the same data
matrix_copy = matrix
matrix_copy1 = matrix.view()
print(matrix_copy)
print(matrix_copy1)
# deep copy
matrix_copy2 = matrix.copy()
print(matrix_copy2)<jupyter_output><empty_output><jupyter_text>More matrix computation<jupyter_code># conditional subsetting
matrix[(1 < matrix[:,0])]
matrix[(1 <= matrix[:,0]) & (matrix[:,0] <= 6)
       & (2 <= matrix[:,1]) & (matrix[:,1] <= 7),]
# row mean
matrix.mean(axis = 0)
# unique values and counts
matrix = np.random.random((3,4), )
uvals, counts = np.unique(matrix, return_counts=True)<jupyter_output><empty_output><jupyter_text>https://www.w3resource.com/python-exercises/numpy/index.php Create a matrix of 5 rows and 3 columns with numbers from 1 to 30.Add 2 to the odd values of the array. Normalize the values in the matrix. Substract the mean and divide by the standard deviation. Create a random array (5 by 3) and compute:    * the sum of all elements    * the sum of the rows     * the sum of the columns<jupyter_code>#Given a set of Gene Ontology (GO) terms and the genes that are associated with these terms find the gene 
#that is associated with the most GO terms

go_terms=np.array(["cellular response to nicotine",
                   "cellular response to hypoxia",
                   "cellular response to lipid"])
genes=np.array(["BAD","KCNJ11","MSX1","CASR","ZFP36L1"])

assoc_matrix = np.array([[1,1,0,1,0],[1,0,0,1,1],[1,0,0,0,0]])

print(assoc_matrix)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12116484659386376, 'fraction_numerical': 0.029467845380481885, 'mean_word_length': 5.893667861409797, 'pattern_counts': {'":': 0, '<': 50, '<?xml version=': 0, '>': 45, 'https://': 8, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '104878', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1973, 'n_tokens_neox': 1879, 'n_words': 540, 'path': 'classes/numpy.ipynb', 'repo_name': 'betteridiot/b575f19', 'size': '5769'}