<jupyter_start><jupyter_text>---  COSC 2673/2793 | Machine Learning     Week 1 Lab Exercises: **Introduction to Python**---We will be using the Python programming language for all lab exercises & assignments in this course. Python is a great general-purpose programming language on its own, but with the help of a few popular libraries (numpy, matplotlib, scikit-learn) it becomes a powerful environment for machine learning.We expect most students to be familiar with python. If you are unfamiliar with Python, you will need to get yourself up to speed. However, don't stress too much. This course doesn't focus on programming, so you won't need to learn too much Python to be able to complete the labs and assignments. The following lab exercise will provide you a quick crash course on both the Python programming language basics so that you can get yourself up to speed. **Note:**  *This tutorial is NOT designed to give you a comprehensive understanding of python and numpy. It is aimed at refreshing your python skills and giving you a start. If you like to learn python from scratch, a LinkedIn learning course is provided at* [Canvas week 0 module](https://rmit.instructure.com/courses/79534/pages/week-0-getting-started?module_item_id=2962379))  Before starting the lab- You need to install anaconda on your local machine or log in to AWS classroom setup for the course (Instructions at [Canvas week 0 module](https://rmit.instructure.com/courses/79534/pages/week-0-getting-started?module_item_id=2962379))- You Need to be familiar with starting an IPython/Jupyter notebook (Instructions at [Canvas week 0 module](https://rmit.instructure.com/courses/79534/pages/week-0-getting-started?module_item_id=2962379))***  Check Python versionIn this course we will be using python 3. You can double-check your Python version at the command line after activating your environment by running shell command `python --version`. In the notebook environment you can run shell commands by adding a `!` in front of the command. Run below cell to get the python version for your notebook.<jupyter_code>!python --version<jupyter_output><empty_output><jupyter_text>Python version should be 3.5 or above. Please check your installation if the above condition is not meat. ___  Basic datatypes and operationsLike most programming languages, python has several data types and supports many mathematical and logical operations.|   |    || --- | --- || Text Type |  str || Numeric Types  |  int, float, complex  || Sequence Types | list, tuple, range || Mapping Type | dict || Set Types | set, frozenset || Boolean Type | bool | | Binary Types | bytes, bytearray, memoryview |  Numeric TypesUnlike c++ (or many other programming languages) we do not have to define the data type of a variable in python. It is inferred automatically and, the data type of a variable can be printed as:<jupyter_code>x = 3
print(type(x))<jupyter_output><empty_output><jupyter_text>We can also perform common mathematical operations on these numeric data types<jupyter_code>print(x)       # Prints "3"
print(x + 1)   # Addition; prints "4"
print(x - 1)   # Subtraction; prints "2"
print(x * 2)   # Multiplication; prints "6"
print(x ** 2)  # Exponentiation; prints "9"
x += 1
print(x)  # Prints "4"
x *= 2
print(x)  # Prints "8"<jupyter_output><empty_output><jupyter_text>Same can be done with floats<jupyter_code>y = 2.5
print(type(y))
print(y, y + 1, y * 2, y ** 2)<jupyter_output><empty_output><jupyter_text>Logical operatorsPython implements all of the usual operators for Boolean logic, but uses English words rather than symbols<jupyter_code>t = True
f = False
print(type(t)) # Prints "<class 'bool'>"
print(t and f) # Logical AND; prints "False"
print(t or f)  # Logical OR; prints "True"
print(not t)   # Logical NOT; prints "False"
print(t != f)  # Logical XOR; prints "True"<jupyter_output><empty_output><jupyter_text>String support in python<jupyter_code>hello = 'hello'    # String literals can use single quotes
world = "world"    # or double quotes; it does not matter.
print(hello)       
print(len(hello))  
hw = hello + ' ' + world  # String concatenation
print(hw)  
hw12 = '%s %s %d' % (hello, world, 12)  # sprintf style string formatting
print(hw12)<jupyter_output><empty_output><jupyter_text>Some useful methods in string class for data pre-processing and cleaning<jupyter_code>s = "hello"
print(s.capitalize())  # Capitalize a string; prints "Hello"
print(s.upper())       # Convert a string to uppercase; prints "HELLO"
print(s.replace('l', '(ell)'))  # Replace all instances of one substring with another;
                                # prints "he(ell)(ell)o"
print('  world '.strip())  # Strip leading and trailing whitespace; prints "world"<jupyter_output><empty_output><jupyter_text>**More detailed descriptions on python strings are provided in** [W3Schools Python Tutorial](https://www.w3schools.com/python/python_strings.asp)---  ContainersPython includes several built-in data types to store collections of data: lists, dictionaries, sets, and tuples  ListsList are like arrays and they are very flexible: - Can be sliced & resized - Can contain elements of different types.<jupyter_code>list1 = [1,2,3,4]  #define and initialize a list
print(list1[0], list1[2])

list1[2] = "three"   # change element to a different data type
print(list1[0], list1[2])

print(list1[-1])     # Negative indices count from the end of the list; prints "2"

list1.append(5)  # Add a new element to the end of the list
print(list1)         # Prints "[3, 1, 'foo', 'bar']"

#print length of list
print("Length of list is: ", len(list1))<jupyter_output><empty_output><jupyter_text>We can also access several elements of a list at ones using slicing.<jupyter_code>list2 = list(range(10))     # range is a built-in function that creates a list of integers
print(list2) 
print(list2[2:4])          # Get a slice from index 2 to 4 (exclusive); 
print(list2[2:])           # Get a slice from index 2 to the end; 
print(list2[:2])           # Get a slice from the start to index 2 (exclusive); 
print(list2[:-1])          # Slice indices can be negative, start to end - 1;
print(list2[1:7:2])        # Get a slice of every other element from index 1 to 6 
print(list2[::-1])         # reverse the list order
list2[2:4] = [8, 9]        # Assign a new sublist to a slice
print(list2)<jupyter_output><empty_output><jupyter_text>Loop over list elements<jupyter_code>temparatures = [30.2, 35.1, 36.6, 33.1]
for temparature in temparatures:
    print(temparature+1)<jupyter_output><empty_output><jupyter_text>If you want access to the index of each element within the body of a loop, use the built-in `enumerate` function:<jupyter_code>temparatures = [30.2, 35.1, 36.6, 33.1]
for inx, temparature in enumerate(temparatures):
    print('Temparature of patient {} is: {:.2f} degrees'.format(inx+1, temparature))<jupyter_output><empty_output><jupyter_text>List comprehensionsCan be used to modify the elements of a list<jupyter_code>nums = [0, 1, 2, 3, 4]
squares = []
for x in nums:
    squares.append(x ** 2)
print(squares)<jupyter_output><empty_output><jupyter_text>In python there is an easier & cleaner way of doing this.<jupyter_code>nums = [0, 1, 2, 3, 4]
squares = [x ** 2 for x in nums]
print(squares)<jupyter_output><empty_output><jupyter_text>Conditions can also be added to list comprehensions.<jupyter_code>nums = [0, 1, 2, 3, 4]
even_squares = [x ** 2 for x in nums if x % 2 == 0]
print(even_squares)<jupyter_output><empty_output><jupyter_text>**More details on list can be found at** [W3Schools Python Tutorial](https://www.w3schools.com/python/python_lists.asp)  DictionariesA dictionary stores a key and value pairs. For example you can use a dictionary to hold the information for a given student.<jupyter_code>d = {'first name': 'John', 'last name': 'Doe', 'age': 27, 'ML marks': 85}  # Create a new dictionary with some data
print(d['first name'])       # Get an entry from a dictionary
print('age' in d)     # Check if a dictionary has a given key
d['AA marks'] = 68     # Set an entry in a dictionary
print(d)    

print(d.get('APT marks', 'N/A'))  # Get an element with a default;
print(d.get('ML marks', 'N/A'))    # Get an element with a default; 
del d['age']         # Remove an element from a dictionary
print(d) # "age" is no longer a key;<jupyter_output><empty_output><jupyter_text>Loop over Dictionaries<jupyter_code>d = {'first name': 'John', 'last name': 'Doe', 'age': 27, 'ML marks': 85}
for key in d:
    value = d[key]
    print('Key `{}` has value `{}`'.format(key, value))<jupyter_output><empty_output><jupyter_text>If you want access to keys and their corresponding values, use the `items` method:<jupyter_code>d = {'first name': 'John', 'last name': 'Doe', 'age': 27, 'ML marks': 85}
for key, value in d.items():
    print('Key `{}` has value `{}`'.format(key, value))<jupyter_output><empty_output><jupyter_text>Similar to list, comprehensions allow you to easily construct dictionaries.<jupyter_code>nums = [0, 1, 2, 3, 4]
even_num_to_square = {x: x ** 2 for x in nums if x % 2 == 0}
print(even_num_to_square)<jupyter_output><empty_output><jupyter_text>SetsSets are useful in representing unordered data. Since sets are unordered, you cannot make assumptions about the order in which you visit the elements of the set. It may not be in the order you put elements in.<jupyter_code>animals = {'red', 'green', 'red', 'blue'}
for idx, color in enumerate(animals):
    print('#%d: %s' % (idx + 1, color))<jupyter_output><empty_output><jupyter_text>Similar to list, comprehensions allow you to easily construct sets.<jupyter_code>from math import sqrt
nums = {int(sqrt(x)) for x in range(30)}
print(nums)<jupyter_output><empty_output><jupyter_text>TuplesA tuple is an (immutable) ordered list of values. A tuple is in many ways similar to a list; one of the most important differences is that tuples can be used as keys in dictionaries and as elements of sets, while lists cannot.<jupyter_code>d = {(x, y): x*y for x in range(10) for y in range(5)}  # Create a dictionary (2D array) with tuple keys
t = (2, 4)        # Create a tuple
print(type(t))    # Prints "<class 'tuple'>"
print(d[t])       # Prints "5"
print(d[(1, 1)])  # Prints "1"<jupyter_output><empty_output><jupyter_text>**More details on tuples can be found at** [W3Schools Python Tutorial](https://www.w3schools.com/python/python_tuples.asp)___  FunctionsPython functions start with the `def` keyword. No need to define the output parameters like c++. Write a function to output if the input number is a whole square of another number.<jupyter_code>def is_whole_square(x):
    
    output = int(round(x ** (1. / 2))) ** 2 == x
    
    return output<jupyter_output><empty_output><jupyter_text>Print whole square number between 0 and 100<jupyter_code>whole_squares = [x for x in range(100) if is_whole_square(x)]
print(whole_squares)<jupyter_output><empty_output><jupyter_text>We can also define functions that has default parameters<jupyter_code>def is_whole_pwr(x, pwr=2):
    
    output = int(round(x ** (1. / pwr))) ** pwr == x
    
    return output
whole_pwr2= [x for x in range(100) if is_whole_pwr(x)]
print(whole_squares)
whole_cube = [x for x in range(100) if is_whole_pwr(x, pwr=3)]
print(whole_cube)<jupyter_output><empty_output><jupyter_text>**More details on functions can be found at** [W3Schools Python Tutorial](https://www.w3schools.com/python/python_functions.asp)___  NumpyNumpy is the core library for scientific computing in Python. It provides a high-performance multidimensional array object, and tools for working with these arrays.A numpy array is a grid of values, all of the same type, and is indexed by a tuple of nonnegative integers. Let's start with one dimensional arrays. A numpy 1D-array can be initiated from a python list and can access the elements using square brackets.  Defining 1d-arrays<jupyter_code>import numpy as np

x = np.array([1,2,3,4,5,6,7])
print (x, x.shape, x.dtype)

# can also change the data type
y = x.astype(np.float)
print (y, y.shape, y.dtype)

# access elements with slicing
print(x[1])
print(x[3:5])<jupyter_output><empty_output><jupyter_text>We can also initialize multidimensional arrays. Multidimensional arrays can have one index per axis.<jupyter_code>b = np.array([[1,2,3,4],[5,6,7,8], [9,10,11,12]])   # 2 dimentional array
print(b.shape)                    
print(b[0, 0], b[0, 1], b[1, 0])  

# access elements with slicing
print(b[:-1, :1])

# we can use `:` to denote whole column or row
print(b[:-1, :])<jupyter_output><empty_output><jupyter_text>Some more important attributes of an ndarray object are:- ndarray.ndim: the number of axes (dimensions) of the array.- ndarray.shape: the dimensions of the array.- ndarray.size: the total number of elements of the array. - ndarray.dtype: an object describing the type of the elements in the array. - ndarray.itemsize: the size in bytes of each element of the array.  Numpy also provide methods to create special type of arrays<jupyter_code>a = np.zeros((2,2))   # Create an array of all zeros
print("a =", a,'\n')              

b = np.ones((1,2))    # Create an array of all ones
print("b =", b,'\n')              

c = np.eye(2)         # Create a 2x2 identity matrix
print("c =", c,'\n')              

d = np.random.random((2,2))  # Create an array filled with random values
print("d =", d,'\n')<jupyter_output><empty_output><jupyter_text>Basic array operationsArithmetic operators on arrays apply elementwise. A new array is created and filled with the result.<jupyter_code>a = np.array([20, 30, 40, 50])
b = np.arange(4)
print(b)

c = a - b
print(c)

d = b**2
print(d)

e = 10 * np.sin(a)
print(e)

f = a < 35
print(f)<jupyter_output><empty_output><jupyter_text>We can also do matrix operations on multidimensional arrays<jupyter_code>A = np.array([[1, 1],[0, 1]])
B = np.array([2, 3])

C = A.dot(B)
print(C)<jupyter_output><empty_output><jupyter_text>Universal functionsNumPy provides familiar mathematical functions such as sin, cos, and exp. Within NumPy, these functions operate elementwise on an array, producing an array as output.<jupyter_code>B = np.random.rand(3,2)
print(B)
print("")

print(np.exp(B))
print("")

print(np.sqrt(B))
print("")

C = np.random.rand(3,2)
print(np.add(B, C))<jupyter_output><empty_output><jupyter_text>Numpy provides many useful functions for performing computations on arrays; one of the most useful is sum<jupyter_code>x = np.array([[1,2],[3,4]])

print(np.sum(x))  # Compute sum of all elements
print(np.sum(x, axis=0))  # Compute sum of each column
print(np.sum(x, axis=1))  # Compute sum of each row

print(np.mean(x))   # Compute mean of all elements;<jupyter_output><empty_output><jupyter_text>**You can find the full list of mathematical functions provided by numpy in the documentation** [Mathematical functions](https://numpy.org/doc/stable/reference/routines.math.html)  IteratingIterating over multidimensional arrays is done with respect to the first axis:<jupyter_code>x = np.random.randint(0,100,size=(3,5))
for row in x:
    print(row)<jupyter_output><empty_output><jupyter_text>If you want to go over each element of a multidimensional array you can use:<jupyter_code>for element in x.flat:
    print(element)<jupyter_output><empty_output><jupyter_text>Shape manipulation An array has a shape given by the number of elements along each axis. The shape of an array can be changed with various commands. Note that the following three commands all return a modified array, but do not change the original array:<jupyter_code>x = np.random.randint(0,100,size=(3,4))
print("x = ", x)

print("Flattened x = ", x.ravel())  # returns the array, flattened

print("Reshaped x = ", x.reshape(6,2))

print("Tanspose of x = ", x.T)<jupyter_output><empty_output><jupyter_text>CopiesSimple assignments make no copy of objects or their data.<jupyter_code>a = np.random.randint(0,100,size=(3,4))
print(a, '\n')

b = a # a and b are two names for the same ndarray object
print(b is a, '\n')

# if a is changed it will change b as well
a[2,2] = 225


print(a, '\n')
print(b, '\n')<jupyter_output><empty_output><jupyter_text>Shallow copyDifferent array objects can share the same data. The `view` method creates a new array object that looks at the same data.<jupyter_code>a = np.random.randint(0,100,size=(3,4))
print(a, '\n')

b = a.view() # a and b share the same data
print(b is a, '\n') # but they are not the same object

# still if a is changed it will change b as well
a[2,2] = 225


print(a, '\n')
print(b, '\n')<jupyter_output><empty_output><jupyter_text>Deep copyThe copy method makes a complete copy of the array and its data.<jupyter_code>a = np.random.randint(0,100,size=(3,4))
print(a, '\n')

b = a.copy() # a and b are different objects
print(b is a, '\n') 

# Now if a is changed it will NOT change b
a[2,2] = 225


print(a, '\n')
print(b, '\n')<jupyter_output><empty_output><jupyter_text>**A more detailed Numpy tutorial is at** [NumPy quickstart](https://numpy.org/devdocs/user/quickstart.html) and [NumPy: the absolute basics for beginners](https://numpy.org/devdocs/user/absolute_beginners.html)___  PlottingFor plotting graphs and images in python we can use `matplotlib` library. Below we have a simple example that uses `matplotlib.pyplot`<jupyter_code>import numpy as np
import matplotlib.pyplot as plt

x = np.arange(-5,5,.1)
y2 = x**2
y3 = x**3

plt.plot(x,y2)
plt.plot(x,y3)

plt.xlabel('x value')
plt.ylabel('y value')
plt.title('Plot functions')
plt.legend(['$y = x^2$','$y = x^3$'])
plt.grid(True)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11623941340305803, 'fraction_numerical': 0.02154265901210709, 'mean_word_length': 4.802770448548813, 'pattern_counts': {'":': 0, '<': 156, '<?xml version=': 0, '>': 155, 'https://': 10, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '38', 'dir': 'jupyter-structured-clean-dedup', 'id': '340284', 'license': 'Apache-2.0', 'n_tokens_mistral': 6023, 'n_tokens_neox': 5718, 'n_words': 2066, 'path': 'ruwanten-rmit-cosc2673_2793-2110-39ae685436a9/labs/week01/Week1_lab_exercises.ipynb', 'repo_name': 'kumarv987/ML-2021-Labs', 'size': '17593'}