<jupyter_start><jupyter_text>An Intro to Python for Machine LearningNadeem  Outline for Today- Basic Python:    - Basic Data Types     - Data Structures (Lists, Dicts, Tuples)    - Control Flow    - All Things Functional    - Saving/Loading Things    - Classes in Python    - Numpy Intro:    - Numpy arrays    - Array math   - Matplotlib intro      To create a new notebook: - Click File - Click New Python 3 Notebook  Python Versions, Packages, and NotebooksThere are two major versions of python, Python2 and Python3. Here we will use **Python 3**.You can check your Python version at the command line by running ```python --version.```For scientific computing, we recommend using the [Anaconda](https://docs.continuum.io/anaconda/) distribution. It comes with most of the basic tools you need (numpy, scipy, matplotlib), and even some ML libraries (tensorflow, theano, scikit-learn). You can download Anaconda distribution [here](https://www.continuum.io/downloads)Anaconda also comes with the Jupyter notebook (used here). You can start a notebook from the command line using ```jupyter notebook```  If you're already familiar with Python 2Here are some important differences between Python 2 and Python 3 that apply to this tutorial. Python 3:- Requires parentheses after print statement- range returns a range instead of a list- reduce function is removed- map, filter return map/filter object, not a list---For the full list of differences, please refer to :https://sebastianraschka.com/Articles/2014_python_2_3_key_diff.html  Basic Python For a more in-depth refresher on Python, consult [The Python Tutorial](https://docs.python.org/2/tutorial/index.html)  Data Types  Numbers<jupyter_code># This is a comment
2+2
x = 6//3       # Type does not need to be declared, Python infers it for you
print ("x, type(x): ", x,",", type(x) ) # This will give an int

x = 17 / 3.0
print ("x, type(x): ", x,",", type(x)) # This will give a float
x = 3
print ("x+1, x*2, x**2: ", x+1,",", x*2,",", x**2)
x += 1
print ("x =", x)<jupyter_output>x+1, x*2, x**2:  4 , 6 , 9
x = 4<jupyter_text>something to watch out for. The limitations of floating point representations<jupyter_code>.1 + .1 +.1 == .3 # because of the way .1 is represented in binary, these are not equal
                  # keep this in mind when you are coding<jupyter_output><empty_output><jupyter_text>Booleans<jupyter_code>x = True
y = False

print( "x and y : ", x and y) # logical and
print ("x or y : ", x or y)  # logical or
print ("not x : ", not x)   # logical not
print ("x != y : ", x != y)  # logical xor
print ("x == y : ", x == y )<jupyter_output>x and y :  False
x or y :  True
not x :  False
x != y :  True
x == y :  False<jupyter_text>Strings<jupyter_code>greet = "Hello World"
print (greet)
print (greet[0])
print (greet[0:3]) #[inclusive,exclusive]
print( greet[3:])
print ("Hello" + ' ' + 'World')
print( len(greet)) # Gives the length of string
print (greet + ' ' + str(42))
print (greet, 42)<jupyter_output>Hello World
H
Hel
lo World
Hello World
11
Hello World 42
Hello World 42<jupyter_text>Some string methods:<jupyter_code>print (greet.lower()) # Convert string to lowercase
print (greet.upper() )# Convert string to uppercase
print (greet)         # Note that this does not change the original string
print (greet.split()) # split based on space
print (greet.split('ll')) # split based on 'll'
#Strings are immutable
greet[0]='a'<jupyter_output><empty_output><jupyter_text>Useful String functions The split function<jupyter_code>x = 'blue,red,green'
print(x.split(",")) #splits data based on input

a,b,c = x.split(",")
print("a:",a)
print("b:",b)
print("c:",c)<jupyter_output>['blue', 'red', 'green']
a: blue
b: red
c: green<jupyter_text>the join function<jupyter_code>s = "-"
print (s.join([a,b,c])) # joins different strings together with common criteria<jupyter_output>blue-red-green<jupyter_text>Data Structures  Lists<jupyter_code>#Python lists are dynamic typed. They can contain anything
arr = ["Hello", True, 42]
print (arr)
print (list(map(lambda x : type(x), arr)))

arr = "The quick brown fox jumps over the lazy dog".split()
print (arr)  # prints the entire list
print (len(arr)) # prints length of the list
print (arr[0]) # Prints the first elemnt
print (arr[3]) # Prints 4th element
print (arr[-1])# Prints the last element ( negative indexes are also supported )
# Modify elements with simple assignment
arr[7] = "sleeping"
print (arr)

# Replacing multiple elements is just as simple
arr[0:4] = ["Nimble","Jack"]
print (arr)

# Delete elements with del
del arr[-2:]
print (arr)

# Add elements with .append()
arr.append("candlestick")
print (arr)
# Join is used to concatenate a list into a string
' '.join(arr)<jupyter_output><empty_output><jupyter_text>Slicing<jupyter_code>#Syntax: [start : end : stride]

a = [8,6,7,5,3,0,9]

print ("a = ", a)      
print ("a[2:5] = ", a[2:5])    # Get a slice from index 2 to 5 (exclusive)
print ("a[:2] = ", a[:2] )      # Get a slice from the start to index 2 (exclusive)
print ("a[2:] = ", a[2:] )    # Get a slice from index 2 to the end
print ("a[:-2] =", a[:-2])      # Slice indices can be negative
b = list(range(16))     # Range is a built in function that gives a list of n integers
print ("b = ", b)
print ("b[0::2] =", b[0::2])     # Add a number to the slice to indicate what to count by
print ("b[1::2] = ", b[1::2])
print ("b[::-2] =", b[::-2])    # Can also count down<jupyter_output>b =  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
b[0::2] = [0, 2, 4, 6, 8, 10, 12, 14]
b[1::2] =  [1, 3, 5, 7, 9, 11, 13, 15]
b[::-2] = [15, 13, 11, 9, 7, 5, 3, 1]<jupyter_text>List Comprehensions<jupyter_code>a = [i**2 for i in range(10)]
print (a)

a = [i for i in range(1,20) if i%3 == 0]
print (a)<jupyter_output>[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
[3, 6, 9, 12, 15, 18]<jupyter_text>List iteration<jupyter_code>l = [chr(n) for n in range(97,107)]

#For loop (Cannot easily get index of elements)
print("Simple for loop:")
for num in l:
    print (num)
    
#Java style loop
print("\nJava style for loop:")
for i in range(len(l)):
    print (i,l[i])
    
#Enumeration loop
print("\nEnumeration loop style for loop:")
for i,num in enumerate(l):
    print (i,num)<jupyter_output>Simple for loop:
a
b
c
d
e
f
g
h
i
j

Java style for loop:
0 a
1 b
2 c
3 d
4 e
5 f
6 g
7 h
8 i
9 j

Enumeration loop style for loop:
0 a
1 b
2 c
3 d
4 e
5 f
6 g
7 h
8 i
9 j<jupyter_text>Dictionary<jupyter_code>d = {"Name":"Harry", "Age":11, "Address":"4 Privet Drive, Little Whinging, Surrey"}
print (d)

# get value based on key
print (d["Name"] )

# change an existing value 
d["Address"] = 'Hogwarts' 
print (d)

# insert new element
d["Godfather"] = "Sirius" 
print (d)

# delete a key-value pair
del d["Godfather"] 
print (d)
# Iterating over dictionary keys
for key in d:
    print( key, ":", d[key])
    
for key,value in d.items():    #this was previously d.iteritems() in python2
    print (key,value)<jupyter_output>Name : Harry
Age : 11
Address : Hogwarts
Name Harry
Age 11
Address Hogwarts<jupyter_text>Tuples<jupyter_code># Like lists, tuples are ordered collections of objects with varying data types
t = (12345, 54321, 'hello!')
print (t)
print (t[2])

# Unlike lists, tuples are immutable
t[0] = 42<jupyter_output>(12345, 54321, 'hello!')
hello!<jupyter_text>Control Flow  if, elif, else<jupyter_code>#if statement, python is all about indentations!
x=3
if x>0:
    print ("I do not like green eggs and ham")
elif x == 0 : 
    print ("I do not like them")
else: #x<=0
    print ("Sam-I-am")<jupyter_output>I do not like green eggs and ham<jupyter_text>Loops<jupyter_code>a = list(range(1,6))
print (a)

# While loops continue until condition is false
while len(a) < 10:
    a.append(0)
print (a)

for element in a:
    if element >= 3:
        print (element)
    element = 5
print (a)

# Alternatively, loop through indices
for i in range(len(a)):
    a[i] += 5
print (a)<jupyter_output>[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5, 0, 0, 0, 0, 0]
3
4
5
[1, 2, 3, 4, 5, 0, 0, 0, 0, 0]
[6, 7, 8, 9, 10, 5, 5, 5, 5, 5]<jupyter_text>Functions<jupyter_code>#You can define your own functions:
def func(x):
    return x*2

#The type is inferred by Python
print ("func(5) = ", func(5))

#Python also makes your functions polymorphic whenever it can
print ("func('string') = ", func('string'))

# Functions can have default parameters
import math
def exp(exponent, base=math.e):
    return base ** exponent

print ("e^4 =", exp(4))
print ("2^4 =", exp(4,2))





# Parameters are passed by reference
def add_key(db, x, y):
    db[x] = y
    x = 5
    

d = {'Name': 'Harry', 'Age': 11, 'Address': 'Hogwarts'}

print (d)
g = "occupation"
add_key(d, g, "Wizard")
print( "d after add_key: ", d)
print (g)<jupyter_output>func(5) =  10
func('string') =  stringstring
e^4 = 54.59815003314423
2^4 = 16
{'Name': 'Harry', 'Age': 11, 'Address': 'Hogwarts'}
d after add_key:  {'Name': 'Harry', 'Age': 11, 'Address': 'Hogwarts', 'occupation': 'Wizard'}
occupation<jupyter_text>A slightly more complicated example<jupyter_code>from random import randint

# a function which implements quicksort
def quicksort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[randint(0,len(arr)-1)]
    left = [x for x in arr if x < pivot]
    middle = [x for x in arr if x == pivot]
    right = [x for x in arr if x > pivot]
    return quicksort(left) + middle + quicksort(right)

print ("quicksort([3,6,8,10,1,2,1,4,7]) = ", quicksort([3,6,8,10,1,2,1,4,7]))
# Lambdas are one-liner functions
# lambda input : output
cube = lambda x : x**3 
print ("cube(3) : ", cube(3))

even = lambda x : x%2 == 0 
print ("even(7) : ", even(7))

add = lambda x,y : x + y 
print ("add(3,7) : ", add(3,7))<jupyter_output>cube(3) :  27
even(7) :  False
add(3,7) :  10<jupyter_text>Map and Filter<jupyter_code>arr = [1,2,3,4,5,6,7,8,9,10]
print ("arr = ", arr)

# map() applies a function to each element in one or more lists
print ("map(cube, arr) : ", list(map(cube, arr)))
print ("map(add, arr, arr[::-1]) : ", list(map(add, arr, arr[::-1])))

# filter() removes elements from a list given some criteria
print ("filter(even, arr) : ", list(filter(even, arr)))<jupyter_output>arr =  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
map(cube, arr) :  [1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]
map(add, arr, arr[::-1]) :  [11, 11, 11, 11, 11, 11, 11, 11, 11, 11]
filter(even, arr) :  [2, 4, 6, 8, 10]<jupyter_text>Saving and loading thingsYou can use pickle package to dump objects as such to the system for re-using them later.<jupyter_code>with open('bottles.txt','w') as fout:
    for i in range(99):
        n = 99
        while n > 0:
            fout.write('%d bottles of beer on the wall, %d bottles of beer...\n'%(n,n))
            n -= 1
            fout.write('Take one down, pass it around, %d bottles of beer on the wall!\n\n'%n)
        fout.write('No more bottles of beer on the wall, no more bottles of beer...\n')
        fout.write('Go to the store and buy some more, 99 bottles of beer on the wall!\n')

# another example
fin = open('test.txt','w')
fin.write("sjkdhbfkjshdbf")
fin.close()
        
with open('bottles.txt','r') as fin:
    for i in range(5):
        print (next(fin).strip())
        
d = {"Name":"Harry", "Age":11, "Address":"4 Privet Drive, Little Whinging, Surrey"}

import pickle
# saving objects
pickle.dump(d, open('db_save_file.pkl',"wb")) # "wb" denotes write binary mode

# loading objects
stored_dict = pickle.load(open('db_save_file.pkl',"rb")) # "rb" denotes read binary mode

d == stored_dict<jupyter_output><empty_output><jupyter_text>you can read more about opening files in python: https://www.pythonforbeginners.com/cheatsheet/python-file-handlingand about pickle:https://docs.python.org/3/library/pickle.html  Classes  Using Python's Object Oriented features will greatly simplify the implementations you have to do in this course.<jupyter_code>class MyClass:
    """A simple example class"""
    x = 7
        
    def __init__(self, y):
        """ Create a new MyClass object """
        self.y = y
        
    def multiply(self):
        """ Give some information.  """
        # class methods must take "self" as an argument 
        # in order to access member variables and methods
        return self.x * self.y

    def swapAndSet(self, z):
        """ Change some information """
        self.x, self.y = self.y, z
    
help(MyClass)
obj = MyClass(13)
print ("obj.f() = ", obj.multiply())

print ("obj.x, obj.y = ", obj.x,",", obj.y)
obj.swapAndSet(5)
print ("obj.x, obj.y = ", obj.x,",", obj.y)

class Complex:
    """ complex(real[, imag]) -> complex number
        Create a complex number from a real part and an optional imaginary part.
        This is equivalent to (real + imag*1j) where imag defaults to 0.
    """
    def __init__(self, real, imag=0): #constructor 
        self.real = real
        self.imag = imag
        
    def add(self, other):
        self.real += other.real
        self.imag += other.imag
    
    def __str__(self):
        """ x.__str__() <=> str(x) """
        return "%.3f + %.3fj"%(self.real, self.imag)
a = Complex(3.0,-4.5)
b = Complex(-2, 2.3)
print ("a = ", a) # __str__ method is automatically called when we print an object
print ("b = ", b)
a.add(b)
print ("a = ", a)<jupyter_output>a =  3.000 + -4.500j
b =  -2.000 + 2.300j
a =  1.000 + -2.200j<jupyter_text>NumpyNumpy is a Python library which provides matrix manipulation capabilities and other standard numerical algorithms. Numpy and smart vectorization is the key to fast scientific computing in Python.Let's import numpy first:<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>Arrays You can create arrays from lists:<jupyter_code>x = np.array([1,2,3,4])
print(x)
print(x.shape)<jupyter_output>[1 2 3 4]
(4,)<jupyter_text>Similarly, you can define a numpy matrix as:<jupyter_code>X = np.array([[1,2,3], [4,5,6]])
print(X)
print(X.shape)
X = np.eye(3)  # Create an array of all zeros
print (X)
X = np.random.random((4,4)) # Create an array filled with random values
print (X)<jupyter_output>[[0.02875174 0.84775922 0.22060069 0.10127288]
 [0.05617433 0.6269804  0.25585163 0.96488842]
 [0.22298656 0.48687699 0.4870639  0.05130113]
 [0.73682848 0.39827448 0.11523237 0.16175099]]<jupyter_text>Indexing Similar to that of lists. Selecting all rows except the first and select all columns:<jupyter_code>#accessing an element; Matrix[column,row]
X[0,0]
#selecting the first row
X[0,:]
#Selecting all rows except the first and select all columns:
X[1:,:]
#select columns 0 and 2
X[:,[0,2]]<jupyter_output><empty_output><jupyter_text>You can also index through boolean formulas:<jupyter_code>X < 0.5 #creates a boolean array
X[X < 0.5] #select all elements that are less than 0.5<jupyter_output><empty_output><jupyter_text>Array Math<jupyter_code>x = np.array([[1,2],[3,4]], dtype=np.float64)
y = np.array([[5,6],[7,8]], dtype=np.float64)

# Elementwise sum; both produce the array
print ("x + y = \n", x + y)
print ("np.add(x, y) = \n", np.add(x, y))<jupyter_output>x + y = 
 [[ 6.  8.]
 [10. 12.]]
np.add(x, y) = 
 [[ 6.  8.]
 [10. 12.]]<jupyter_text>Use the dot function to compute inner products of vectors, to multiply a vector by a matrix, and to multiply matrices.<jupyter_code>print ("x.dot(y) = \n", x.dot(y))
print ("np.dot(x,y) = \n", np.dot(x, y))<jupyter_output>x.dot(y) = 
 [[19. 22.]
 [43. 50.]]
np.dot(x,y) = 
 [[19. 22.]
 [43. 50.]]<jupyter_text>Of course, dimensions must match<jupyter_code>x2 = np.array([[1,2], [1,2]])
print ("np.shape(x2) = ", np.shape(x2))
y2 = np.array([[1,2,3],[1,2,3],[1,2,3]])
print ("np.shape(y2) = ", np.shape(y2))
np.dot(x2,y2)
# calculate transpose and inverse
x_transp = x.T
print ("x = \n", x)
print ("x_transp = \n",x_transp)
x_inv = np.linalg.inv(x)
print ("x_inv = \n", x_inv)<jupyter_output>x = 
 [[1. 2.]
 [3. 4.]]
x_transp = 
 [[1. 3.]
 [2. 4.]]
x_inv = 
 [[-2.   1. ]
 [ 1.5 -0.5]]<jupyter_text>And now you have enough to do linear regression!  MatplotLibMatplotlib is a plotting library. As usual, we import it first<jupyter_code>import matplotlib.pyplot as plt
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Use the ```plot``` function to plot the 2D data.<jupyter_code>plt.title("My Plot")
plt.plot([1,2,3,4])
plt.ylabel('some numbers')
plt.show()<jupyter_output><empty_output><jupyter_text>The format of plot function is (x-axis values, y-axis values, style)<jupyter_code>plt.plot([1,2,3,4], [1,4,9,16], 'ro') # Here r means red, o means circle
plt.axis([0, 6, 0, 20]) # changes the default axis length
plt.show()<jupyter_output><empty_output><jupyter_text>You can also plot multiple curves<jupyter_code>t = np.arange(0., 5., 0.2)

plt.plot(t, t, 'r--')
plt.plot(t, t**2, 'bs')
plt.plot(t, t**3, 'g^')
plt.xlabel('x axis label')
plt.ylabel('y axis label')
plt.title('Curves')
plt.legend(['t', 't**2', 't**3'])
plt.show()<jupyter_output><empty_output><jupyter_text>You can plot different curves in the same figure using the ```subplot``` function.<jupyter_code>def f(t):
    return np.exp(-t) * np.cos(2*np.pi*t)

t1 = np.arange(0.0, 5.0, 0.1)
t2 = np.arange(0.0, 5.0, 0.02)

plt.figure(1)
plt.subplot(211)
plt.plot(t1, f(t1), 'bo', t2, f(t2), 'k')

plt.subplot(212)
plt.plot(t2, np.cos(2*np.pi*t2), 'r--')

plt.show()<jupyter_output><empty_output><jupyter_text>More readinghttp://matplotlib.org/users/pyplot_tutorial.html  Practice Problems  1. Return the maximum number and the index of that number from an array of numbers<jupyter_code>data = [123,-23,454,34,766,-2,0,25,-43,65]

max_value = max(data)
max_index = data.index(max_value)

print("the max value is {0} which is found at index {1}".format(max_value,max_index))<jupyter_output>the max value is 766 which is found at index 4<jupyter_text>2. Plot a sin function as well as noise corrupted data from this sin function (as seen in class)<jupyter_code>def sin_function(t):
  
  return np.sin(2*np.pi*t)

t1 = np.arange(0,1,0.1)

y = sin_function(t1)
noise = np.random.normal(0,0.1,10)

data = y + noise # data observed

# origional data
x_coords = np.arange(0,1,0.01)
origional_data = sin_function(x_coords)

plt.plot(t1,data,"bo") # plot observed data
plt.plot(x_coords,origional_data) #plot origional data
plt.title("Plot of noise corrupted data") #give title
plt.legend(["obeserved data",'true underlying function'])
plt.show()<jupyter_output><empty_output><jupyter_text>3. Determine if an array is monotonic.Example:- [1,2,3,3,3,3,3,4,5,6,7] is monotonic (increasing)- [7,6,5,4,3,3,3,3,3,2,1] is monotonic (decreasing)- [1,5,4,3,3,3,3,3,2,6,7] is not monotonic<jupyter_code>def isMonotonic(arr):
    behaviour = None
    for i in range(len(arr)-1):
        b = arr[i] - arr[i+1]
        if b > 0:
            b = 1
        if b < 0:
            b = -1        
        
        if b != 0:
            if behaviour == None:
                behaviour = b
            else:
                if behaviour != b:
                    return False
    return True
print (isMonotonic([1,5,4,3,3,3,3,3,2,6,7]))
print (isMonotonic([7,6,5,4,3,3,3,3,3,2,1]))
print (isMonotonic([1,1,1,1,1,1,1]))<jupyter_output>False
True
True<jupyter_text>4. Hard(er) problem for plotting using matplotlib Plot the decision boundary for a circle of radius 2<jupyter_code>import matplotlib.colors as colors

size_of_map = np.arange(-3,3,0.01)
xx, yy = np.meshgrid(size_of_map,size_of_map) # gives a rectangular grid out of 
                                              # input values

# our data is generated randomly from the uniform random distribution
x0 = np.random.uniform(-3,3,100)
x1 = np.random.uniform(-3,3,100)

# the underlying classification function, note that it has small gaussian noise
def true_function(x0,x1):
  
  return x0**2 + x1**2 + np.random.normal(0,1) <= 4 

y = true_function(x0,x1) #the true predictions for our data


# our classifier, just the circle without the gaussian noise
def classifier(x0,x1):
  
  return x0**2 + x1**2 <= 4
  
z = classifier(xx,yy) # our classifier applied to the entire grid 


cm_bright = colors.ListedColormap(['#FF0000', '#0000FF']) #just colors
plt.contourf(xx,yy,z,alpha=0.4,cmap=cm_bright) #plot the contours using our classifier results
plt.scatter(x0,x1,c=y,s=20,cmap = cm_bright) #scatter plot of true data

# labelling the plot
plt.xlabel("x0")
plt.ylabel("x1")
plt.title("Classifier boundries vs true data results")
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1530467179763714, 'fraction_numerical': 0.048482768763174665, 'mean_word_length': 4.0, 'pattern_counts': {'":': 7, '<': 136, '<?xml version=': 0, '>': 133, 'https://': 6, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '37', 'dir': 'jupyter-structured-clean-dedup', 'id': '151893', 'license': 'MIT', 'n_tokens_mistral': 8247, 'n_tokens_neox': 7617, 'n_words': 2402, 'path': 'Tutorial1/Tutorial 1-Intro to Python.ipynb', 'repo_name': 'NadeemWard/Intro-RL-course-tutorials', 'size': '20399'}