<jupyter_start><jupyter_text>---     Department of Data ScienceCourse: Tools and Techniques for Data Science---Instructor: <NAME>, Ph.D.     Lecture 2.12  _Advance-Functions.ipynb_  Review of Python Functions   Learning agenda of this notebook1. Anonymous / Lambda Functions2. Using lambda function as argument to other functions3. Using Lambda Function with built-in `map()` function4. Using Lambda Function with built-in `filter()` function5. Using Lambda Function with built-in `reduce()` function6. Using Lambda Function with built-in `sorted()` function7. Bonus    - The `zip()` function    - Iterators and Generators  1. Lambda / Anonymous FunctionsThe syntax of defining a lambda function is:**```lambda [arg1 [,arg2,.....argn]]:expression```**   a. Example 1: A function that is passed one argument and it returns its square- Let us do this example step by step to completely understand the process of writing lambda functions<jupyter_code>def square1(num):
    result = num**2
    return result
rv = square1(5)
rv<jupyter_output><empty_output><jupyter_text>**Let us try to shrink the above function**<jupyter_code>def square2(num):
    return num**2
rv = square2(5)
rv<jupyter_output><empty_output><jupyter_text>**Although not a good programming style, however, we can write this in a single line**<jupyter_code>def square3(num): return num**2
rv = square3(5)
rv<jupyter_output><empty_output><jupyter_text>**This is the form a function that a lambda expression intends to replicate. A lambda expression can then be written as:**<jupyter_code>lambda num: num**2<jupyter_output><empty_output><jupyter_text>Note how we get a function back. We can assign this function to a label:<jupyter_code>square4 = lambda num: num**2
print(square4)      
print(type(square4))
rv = square4(5)
rv<jupyter_output><empty_output><jupyter_text>b. Example 2: A function that is passed two arguments and it returns their sum```def mysum2(a, b):   return a+brv = mysum2(5, 7)rv```<jupyter_code>mysum2 = lambda a, b: a + b   

rv = mysum2(5.3, 7)
rv<jupyter_output><empty_output><jupyter_text>c. Example 3: A function that is passed three arguments and it returns their sum```def mysum3(a, b, c):   return a+b+crv = mysum3(5, 7, 9)rv```<jupyter_code>mysum3 = lambda a, b, c: a + b + c  

rv = mysum3(5.5, 6.3, 2.7)         # return type corresponds to the expression
rv<jupyter_output><empty_output><jupyter_text>d. Example 4: A function that is passed one argument and it returns True if it is even and False otherwise<jupyter_code>even = lambda x: x % 2 == 0
rv = even(3)         # return type corresponds to the expression
rv<jupyter_output><empty_output><jupyter_text>e. Example 5: A function that is passed a string and returns the first character of that string<jupyter_code>func = lambda s: s[0]
rv = func("Hello")
rv<jupyter_output><empty_output><jupyter_text>f. Example 6: A function that is passed a string, it returns the reverse of that string<jupyter_code>func = lambda s: s[::-1]

rv = func("Hello")
rv<jupyter_output><empty_output><jupyter_text>g. Example 7: A function that is passed a list and it returns the length of the list<jupyter_code>func = lambda arg: len(arg)

list1 = ['Learning', 'is', 'fun', 'with', 'arif']
rv = func(list1)
rv<jupyter_output><empty_output><jupyter_text>>**The real usage of Python Lambda functions is actually passing them as arguments to other functions like `map()`,`filter()` and `reduce()`**  2. Using Lambda Function as Argument to other Functions **Three regular function definitions and their calling convention**<jupyter_code># Some basic functions that receives two arguments and return their sum, diff, mul
def myadd(a, b):
    return a + b

def mysub(a, b):
    return a - b

def mymul(a, b):
    return a * b

# Calling above functions
rv1 = myadd(8,2)
rv2 = mysub(8, 2)
rv3 = mymul(8, 2)
rv1, rv2, rv3<jupyter_output><empty_output><jupyter_text>**We can write above three functions as lambda functions**<jupyter_code>myadd = lambda a, b: a+b
mysub = lambda a, b: a-b
mymul = lambda a, b: a*b<jupyter_output><empty_output><jupyter_text>**Now, let us write a calculator function that other than receiving two arguments, also receives a function name which specifies the operation that needs to be performed on the arguments**<jupyter_code>def mycalc(op, a, b):
    return op(a,b)
rv1 = mycalc(myadd, 8, 2)
rv2 = mycalc(mysub, 8, 2)
rv3 = mycalc(mymul, 8, 2)

rv1, rv2, rv3<jupyter_output><empty_output><jupyter_text>**A more elegant way of writing above code**<jupyter_code>rv1 = mycalc(lambda a, b: a + b, 8, 2)
rv2 = mycalc(lambda a, b: a - b, 8, 2)
rv3 = mycalc(lambda a, b: a - b, 8, 2)

rv1, rv2, rv3<jupyter_output><empty_output><jupyter_text>3. Using Lambda Function as Argument to built-in `map()` Function- The ```map(aFunction, *iterables)``` function simply returns a map object after applying  `aFunction()` to all the elements of `iterable(s)`. Later you can type cast the map object to appropriate data structure- The original iterable(s) remains unchanged.<jupyter_code>help(map)<jupyter_output>Help on class map in module builtins:

class map(object)
 |  map(func, *iterables) --> map object
 |  
 |  Make an iterator that computes the function using arguments from
 |  each of the iterables.  Stops when the shortest iterable is exhausted.
 |  
 |  Methods defined here:
 |  
 |  __getattribute__(self, name, /)
 |      Return getattr(self, name).
 |  
 |  __iter__(self, /)
 |      Implement iter(self).
 |  
 |  __next__(self, /)
 |      Implement next(self).
 |  
 |  __reduce__(...)
 |      Return state information for pickling.
 |  
 |  ----------------------------------------------------------------------
 |  Static methods defined here:
 |  
 |  __new__(*args, **kwargs) from builtins.type
 |      Create and return a new object.  See help(type) for accurate signature.<jupyter_text>a. Example 1: **Given a list of numbers, suppose we want to create a new list in which every element is the square every item of the given list** **Option 1: We can do this using a simple loop**<jupyter_code>mylist = [5, 7, 2, 6, 9]

mylist_squared = []    # create an empty list
for a in mylist:
    mylist_squared.append(a**2)   # append new item at the end of the newly created list

print("Original list: ", mylist)
print("List with items squared: ", mylist_squared)<jupyter_output>Original list:  [5, 7, 2, 6, 9]
List with items squared:  [25, 49, 4, 36, 81]<jupyter_text>**Option 2: We can do this using `map()` function and passing an appropriate regular function as its first argument**<jupyter_code>mylist = [5, 7, 2, 6, 9]

def sqr(x):
    return x ** 2


map_object = map(sqr, mylist)
mylist_squared = list(map_object)

print("Original list: ", mylist)
print("List with items squared: ", mylist_squared)<jupyter_output>Original list:  [5, 7, 2, 6, 9]
List with items squared:  [25, 49, 4, 36, 81]<jupyter_text>- We passed a user defined function `sqr(x)`, to the `map` function, along with the list of items on which to apply that function- `map()` function calls `sqr()` function on each list item and collects all the return values into a map object, which is type casted to a list- Since `map()` expects a function to be passed in, so this is where we can also use lambda functions as shown below **Let us use Lambda Function as key argument to `map()` function to perform the above task**<jupyter_code>mylist = [5, 7, 2, 6, 9]

map_object = map(lambda x: x ** 2 , mylist)

mylist_squared = list(map_object)

print("Original list: ", mylist)
print("List with items squared: ", mylist_squared)<jupyter_output>Original list:  [5, 7, 2, 6, 9]
List with items squared:  [25, 49, 4, 36, 81]<jupyter_text>**Let us do this with List comprehension**<jupyter_code>mylist = [5, 7, 2, 6, 9]
newlist = [i**2 for i in mylist]
newlist<jupyter_output><empty_output><jupyter_text>b. Example 2: **Given a list of numbers, suppose we want to create a new list in which every element is the remainder once the original list element is divided by 5**<jupyter_code>mylist = [74, 85, 14, 23, 56, 32, 45 ]

map_object = map(lambda num: num%5, mylist)

remainders = list(map_object)

print("Original list: ", mylist)
print("List of remainders: ", remainders)<jupyter_output>Original list:  [74, 85, 14, 23, 56, 32, 45]
List of remainders:  [4, 0, 4, 3, 1, 2, 0]<jupyter_text>c. Example 3: **Suppose we want to add two lists**<jupyter_code>mylist1 = [4, 8, 3, 2]
mylist2 = [3, 1, 2, 6]

result = map(lambda a, b: a + b, mylist1, mylist2) #two arguments are passed to lambda func (one from each list)
result = list(result)

print("Sum of the two lists: ", result)<jupyter_output>Sum of the two lists:  [7, 9, 5, 8]<jupyter_text>d. Example 4: **Given a list of strings, suppose we want to create another list that contains the length of each string in the list**<jupyter_code>list1 = ('Mujahid', '<NAME>', 'Kakamanna', 'Maaz')

result = map(lambda a: len(a), list1)
result = list(result)

print("Length of Strings in list1: ", result)<jupyter_output>Length of Strings in list1:  [7, 9, 9, 4]<jupyter_text>4. Using Lambda Function as Argument to built-in `filter()` Function```filter(function or None, iterable)```- The `filter()` function offers a convenient way to filter out all the elements of an iterable, for which the function returns true.- If function argument is None, return the items that are itself True.- The filter object contains only those items of iterable for which  `function(item)` returns True. - The original iterable remains unchanged. - The filter object can be converted to a list using the `list()` function  a. Example 1: **A very basic usage of `filter()`, that returns the True elements of a list**- In Python, the following objects are considered false:    >- Constants like `None` and `False`    >- Numeric types having values: 0, 0.0, 0j    >- Empty sequences and collections like `""`, `()`, `[]`, `{}`, `set()`, and `range(0)`<jupyter_code>mylist = [5, 0, -3, {}, False, 0.0, True, 9, 0j, (), None, 8]

result = filter(None, mylist)
print(result)
print(list(result))<jupyter_output><filter object at 0x7fd8c0644ee0>
[5, -3, True, 9, 8]<jupyter_text>b. Example 2: **Suppose we want to extract even numbers from a list**<jupyter_code>numbers = [1, 5, 4, 6, 8, 11, 3, 12]

result = filter(lambda x: x%2 == 0 , numbers)

result = list(result)

print("Even numbers in the list are: ", result)<jupyter_output>Even numbers in the list are:  [4, 6, 8, 12]<jupyter_text>**Let us do it using List Comprehension**<jupyter_code>numbers = [1, 5, 4, 6, 8, 11, 3, 12]
newlist = [i for i in numbers if i%2 == 0]
newlist<jupyter_output><empty_output><jupyter_text>c. Example 3: **Suppose we want to extract negative numbers from a list**<jupyter_code>numbers = [25, -3, -8, 17, 3, 8, -3, 6, -7, 0]

result = filter(lambda x: x<0, numbers)

result = list(result)

print("Negative numbers in the list are: ", result)<jupyter_output>Negative numbers in the list are:  [-3, -8, -3, -7]<jupyter_text>d. Example 4: **Suppose we want to extract vowels from a list of alphabets**<jupyter_code>characters = ['i', 'z', 'b', 'a', 'd', 'f', 't', 'e','w', 'x']

vowels = ['a', 'e', 'i', 'o', 'u']
    
result = filter(lambda x: x in vowels, characters)

result = list(result)    
print("Vowels in the list are: ", result)<jupyter_output>Vowels in the list are:  ['i', 'a', 'e']<jupyter_text>5. Using Lambda Function as Argument to built-in `reduce()` Function- The `reduce()` works differently than `map()` and `filter()`. It does not return a new iterable based on the function and iterable we've passed. Instead, it returns a single value.```reduce(func, sequence[, initial])``` - If seq = `[ s1, s2, s3, ... , sn ]`, then calling `reduce(func, seq)` works like this:    - Apply `func` argument to the first two items in the iterable and generate a partial result, i.e. `func(s1,s2)`    - The list on which `reduce()` works looks now like this: `[ func(s1, s2), s3, ... , sn ]`    - In the next step the function will be applied on the previous result and the third element of the list, i.e. `func(func(s1, s2),s3)`    - The list looks like this now: `[ function(function(s1, s2),s3), ... , sn ]`    - It continues like this until just one element is left and return this element as the result of `reduce()`- If initial is present, it is placed before the items of the sequence in the calculation, and serves as a default when the sequence is empty. The optional argument `initial` when provided is used as the 0th element- In Python 3.x, if you need to use `reduce()`, then you first have to import the function into your current scope using an import statement in one of the following ways:    - `import functools` and then use fully-qualified names like functools.reduce().    - `from functools import reduce` and then call reduce() directly.      a. Example 1: **Suppose given a list of numbers and we want to get the accumulative sum of all the numbers in that list** **Option 1: We can do this using a simple loop**<jupyter_code>numbers = [47, 11, 42, 13]
total = 0

for i in numbers:
    total += i

total<jupyter_output><empty_output><jupyter_text>**Option 2: We can do this using `reduce()` function and passing an appropriate regular function as its first argument. Remember this has to be a function that receives two arguments**<jupyter_code>#Example: Note the call to reduce(), 
# applies myadd() to the items in the numbers list to compute their accumulative sum    

from functools import reduce
def myadd(a,b):
    return a+b

numbers = [47, 11, 42, 13]

rv = reduce(myadd, numbers)
rv<jupyter_output><empty_output><jupyter_text>**Option 3: Let us use Lambda Function as first argument to `reduce()` function to perform the above task**<jupyter_code># Example: Use lambda function for above task
from functools import reduce
numbers = [47, 11, 42, 13]

rv = reduce(lambda x,y: x+y, numbers)
rv<jupyter_output><empty_output><jupyter_text>**Let us now understand the initial argument to `reduce()` function**```reduce(func, sequence[, initial])```<jupyter_code># Example: Use of initial argument of reduce() function
from functools import reduce
numbers = [47, 11, 42, 13]

rv = reduce(lambda x,y:x+y, numbers, 10)
rv<jupyter_output><empty_output><jupyter_text>b. Example 2: **Multiplying all numeric values of a list with each other**<jupyter_code>from functools import reduce
numbers = [1, 2, 3, 4, 5]

rv = reduce(lambda x,y: x*y, numbers)
rv<jupyter_output><empty_output><jupyter_text>c. Example 3: **Finding minimum or maximum number from a list of numbers**<jupyter_code>numbers = [-10, -20, -93, -4, -5]


rv1 = reduce(lambda a, b: a if a > b else b, numbers, 0)


rv1
from functools import reduce
numbers = [10, 20, 93, 4, 5]


rv1 = reduce(lambda a, b: a if a < b else b, numbers)
rv2 = reduce(lambda a, b: a if a > b else b, numbers)

rv1, rv2<jupyter_output><empty_output><jupyter_text>d. Example 4: **Checking if ALL values in an iterable are true**<jupyter_code>from functools import reduce
mylist = [0, 0, 1, 0, 0]

rv = reduce(lambda a, b: bool(a and b), mylist)
rv<jupyter_output><empty_output><jupyter_text>e. Example 5: **Checking if ANY value in an iterable is true**<jupyter_code>from functools import reduce
mylist = [0, 0, 1, 0, 0]

rv = reduce(lambda a, b: bool(a | b), mylist)
rv<jupyter_output><empty_output><jupyter_text>6. Using Lambda Function as Argument to built-in `sorted()` Function- We have already seen the use of `sorted()` function in our previous session of Tuples.- The only required argument to `sorted()` function is an iterable. - It sorts the items of the given iterable in ascending order (by default) and returns the sorted iterable as a list, without modifying the original iterable.```sorted(iterable, key=None, reverse=False)```We also have seen the use of `key` argument, where we can pass a function that is applied once to each element of the iterable before sorting it.  a. Example 1: **Consider a tuple of strings `('abcz', 'xyza', 'bas', 'arif')`. If we pass this tuple to `sorted()` function, it will sort the list alphabatically.**<jupyter_code>t1 = ('abcz', 'xyza', 'bas', 'arif')

rv = sorted(t1)

print("Sorted tuple: ", rv)
print("Original tuple remains as such: ", t1)<jupyter_output>Sorted tuple:  ['abcz', 'arif', 'bas', 'xyza']
Original tuple remains as such:  ('abcz', 'xyza', 'bas', 'arif')<jupyter_text>**Suppose, we want to sort the above tuple by last character of strings within the tuple so that the output is like : `('xyza', 'arif', 'bas', 'abcz')`. We can do this by passing an appropriate regular function to `key` argument of the `sorted()` function**<jupyter_code>def last(s):
    return s[-1]

t1 = ('abcz', 'xyza', 'bas', 'arif')
rv = sorted(t1, key=last)

print("Sorted tuple: ", rv)
print("Original tuple remains as such: ", t1)<jupyter_output><empty_output><jupyter_text>**Let us use Lambda Function as `key` argument to `sorted()` function to perform the above task**<jupyter_code>t1 = ('abcz', 'xyza', 'bas', 'arif')

rv = sorted(t1, key = lambda arg : arg[-1])

print("Sorted tuple: ", rv)
print("Original tuple remains as such: ", t1)<jupyter_output>Sorted tuple:  ['xyza', 'arif', 'bas', 'abcz']
Original tuple remains as such:  ('abcz', 'xyza', 'bas', 'arif')<jupyter_text>b. Example 2: **Suppose given a list in which each element is a two valued tuple `[(4, 30), (6, 15), (1, 25), (9, 8)]`. If we call `sorted()` function on this list it will sort it based on the first element of each tuple.** **Simple sorting of above list using `sorted()` function, will sort by first element**<jupyter_code>mylist = [(4, 30), (6, 15), (1, 25), (9, 8)]


mylist_sorted = sorted(mylist)

mylist_sorted<jupyter_output><empty_output><jupyter_text>**But we want to sort the above list based on the second element of each tuple, so that the output is like: `[(9, 8), (6, 15), (1, 25), (4, 30)]`. We can do this by passing an appropriate regular function to `key` argument of the `sorted()` function**<jupyter_code>mylist = [(4, 30), (6, 15), (1, 25), (9, 8)]

def func(item):
    return item[1]

mylist_sorted = sorted(mylist, key = func)

mylist_sorted<jupyter_output><empty_output><jupyter_text>**Let us use Lambda Function as `key` argument to `sorted()` function to perform the above task**<jupyter_code>mylist = [(4, 30), (6, 15), (1, 25), (9, 8)]

mylist_sorted = sorted(mylist, key = lambda element:element[1])

mylist_sorted<jupyter_output><empty_output><jupyter_text>7.  Bonus  a. The `zip()` Function- The `zip(*iterables)` function returns an iterator of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. - The iterator stops when the shortest input iterable is exhausted. - With a single iterable argument, it returns an iterator of 1-tuples. With no arguments, it returns an empty iterator.<jupyter_code>x = [1,2,3]
y = [4,5,6]

zip_object = zip(x ,y)

mylist = list(zip_object)
mylist
x = [1, 2, 3, 4]
y = [5, 6, 7, 8]
z = [9, 10, 11, 12]

zip_object = zip(x,y,z)

mylist = list(zip_object)
mylist<jupyter_output><empty_output><jupyter_text>**The iterator stops when the shortest input iterable is exhausted.**<jupyter_code>x = [1,2,3]
y = [4,5,6,7,8]

zip_object = zip(x,y)

mylist = list(zip_object)
mylist<jupyter_output><empty_output><jupyter_text>**With a single iterable argument, it returns an iterator of 1-tuples.**<jupyter_code>x = [1, 2, 3, 4]

zip_object = zip(x)

mylist = list(zip_object)
mylist<jupyter_output><empty_output><jupyter_text>**With no arguments, it returns an empty iterator.**<jupyter_code>zip_object = zip()

mylist = list(zip_object)
mylist<jupyter_output><empty_output><jupyter_text>b. Iterators and Generators**Iterators:**- In our previous session of `for` loops, we discussed the concept of `Iterator` object, which is used to iterate over iterable objects like lists and tuples.- We also discussed the `iter()` function, which is passed an iterable (list, tuple, ...) and it returns an iterator for that iterable object. - We also discussed the `next()` function, which is passed the iterator object, and each time it is called it returns the next item of that iterator object. **Example: Iterator Objects**<jupyter_code>mylist = ['banana', 'mango', 'grapes']
a = iter(mylist)
a
print(next(a))
print(next(a))
print(next(a))
next(a)<jupyter_output><empty_output><jupyter_text>**Generators:**- Today let me tell you as how to write a generator function in Python. A generator is a function that can send back a value and then later resume its execution from where it left off. - A generator function allows us to generate a sequence of values over time. - The main difference between a regular function and a generator function is that instead of using a `return` statement, the generator function uses the `yield` statement- So once a generator function is called, it don't actually return a value and then exit, rather it automatically suspend and resume its execution and state around the last point of value generation.  **Example 1: Writing a Hello World Generator Function to understand the `yield` keyword**<jupyter_code>def mygenerator():
    print('First item')
    yield 10

    print('Second item')
    yield 20

    print('Last item')
    yield 30
gen = mygenerator() 
gen
print(next(gen))
print(next(gen))
print(next(gen))<jupyter_output>Last item
30<jupyter_text>**Example 2: Writing a Generator Function that returns cubes of numbers**<jupyter_code>def gencubes(n):
    for num in range(n):
        yield num**3
for x in gencubes(5):
    print(x)<jupyter_output>0
1
8
27
64
{'fraction_non_alphanumeric': 0.12387324598085679, 'fraction_numerical': 0.025044140879100455, 'mean_word_length': 4.795099623047927, 'pattern_counts': {'":': 0, '<': 187, '<?xml version=': 0, '>': 192, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '49', 'dir': 'jupyter-structured-clean-dedup', 'id': '574940', 'license': 'BSL-1.0', 'n_tokens_mistral': 7504, 'n_tokens_neox': 7055, 'n_words': 2806, 'path': 'data-science-master/Section-2-Basics-of-Python-Programming/Lec-2.12-Advanced-Functions/Advance-Functions.ipynb', 'repo_name': 'Hamid-Ali-99/Python_Just_Python', 'size': '21537'}