<jupyter_start><jupyter_text>ListsLists are the most versatile data structure in python. What is a list exactly? A list is a type of **sequence** in python. The sequence is the most basic data type in python! Other sequence types are strings (yeah, strings are also sequences) and tuples. This is why we will see that lists and strings have **a lot** in common.If both lists and strings are sequences though, what's the difference between them? Strings are sequences of characters, while lists can be sequences of anything we want! That's why we said that lists are so versatile.A list is a sequence of objects enclosed by square brackets `[]`. Let's dive in.<jupyter_code>from __future__ import print_function<jupyter_output><empty_output><jupyter_text>We'll first create a few lists.<jupyter_code># Empty lists
L = []
L = list()
# Populated lists
L1 = ['a','b','c','d','e']
L2 = [1, 2, 3, 4, 5]

print('L = ', L)
print('L1 = ', L1)
print('L2 = ', L2)<jupyter_output>L =  []
L1 =  ['a', 'b', 'c', 'd', 'e']
L2 =  [1, 2, 3, 4, 5]<jupyter_text>IndexingIndexing works the same way it did with strings. We can refer to a single element in a list by placing the element's index inside of square brackets, immediately after the list's name.The list's elements are indexed as integers **starting from zero** to (n-1), where n is the length of the list.For example, the *i*-th element of the list `ls` has an index of *i-1*. So in order to access it we need to type:```pythonls[i-1]```By using negative indices we can access elements of the list by starting from the last one.<jupyter_code>print('First element: ', L1[0]) # first element
print('Last element: ', L1[-1]) # last element<jupyter_output>First element:  a
Last element:  e<jupyter_text>Slicing> Slicing is the operation in which we keep only a **subset** of the list.We can slice a list (`ls`) by using the following syntax:```pythonls[a:b:c]```- **a**: the index of the **starting point** of our slicing operation.- **b**: the index of the **ending point** (the final element in the slice will have the index of *b-1*).- **c**: the **step** of the operation (optional, *default=1*).<jupyter_code>print('First two elements:                             L1[:2]    = ', L1[:2])
print('Elements from index 1 and onward:               L1[1:]    = ', L1[1:])
print('Last three elements:                            L1[-3:]   = ', L1[-3:])
print('Elements with indexes 1 to 3:                   L1[1:4]   = ', L1[1:4])
print('All elements of L1:                             L1[:]     = ', L1[:])
print('Elements from indices 2 to 4 with a step of 2:  L1[2:5:2] = ', L1[2:5:2])
print('Get every other item, starting with the first:  L1[::2]   = ', L1[::2])
print('Get every other item, starting with the second: L1[1::2]  = ', L1[1::2])<jupyter_output>First two elements:                             L1[:2]    =  ['a', 'b']
Elements from index 1 and onward:               L1[1:]    =  ['b', 'c', 'd', 'e']
Last three elements:                            L1[-3:]   =  ['c', 'd', 'e']
Elements with indexes 1 to 3:                   L1[1:4]   =  ['b', 'c', 'd']
All elements of L1:                             L1[:]     =  ['a', 'b', 'c', 'd', 'e']
Elements from indices 2 to 4 with a step of 2:  L1[2:5:2] =  ['c', 'e']
Get every other item, starting with the first:  L1[::2]   =  ['a', 'c', 'e']
Get every other item, starting with the second: L1[1::2]  =  ['b', 'd']<jupyter_text>Range> With the range function we can quickly get a list of all consecutive integers between two values, or a list of all integers between two values with a certain step.Syntax is:```pythonrange(a,b,c)```- **a**: **starting point** (optional, *default=0*).- **b**: **ending point** (last integer of the range is *b-1*).- **c**: **step** (optional, *default=1*).While in python2 `range()` would return a *list*, in python3 `range()` returns an object called a **generator**. We'll see in the future what generators are. For the time being we will simple cast them as lists.<jupyter_code>print('range(10) = ', list(range(10)))
# this creates a list of integers ranging from 0 to the number before what we entered as a parameter.
# range(x) essentially creates a list of all integers in [0,x)
print('range(5,10) = ', list(range(5,10)))
# like before, range(x,y) creates a list of all integers in [x,y)
print('range(1,20,2) = ', list(range(1,20,2)))
# this creates a list of integers in the range defined by the first two numbers,
# but with a step of the third one.<jupyter_output>range(10) =  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
range(5,10) =  [5, 6, 7, 8, 9]
range(1,20,2) =  [1, 3, 5, 7, 9, 11, 13, 15, 17, 19]<jupyter_text>Range **only works on integers**. If we want a range of floats we need to create a custom function (we'll learn how to do that on a later tutorial).  Updating list elementsWe can change (update) the value of any element in a list by simple assignment. If we want to change multiple values we need to make sure we slice as many values as we assign.<jupyter_code>print('Before changes: L2 =', L2)
L2[2] = 6 # we update the 3rd element in our list
print('Single update: L2 =', L2)
L2[1:4] = [22, 33, 44] # update the 2nd, 3rd and 4th elements
print('Multiple update 1: L2 =', L2)
L2[1:4] = 222, 333, 444 # same thing without the list
print('Multiple update 2: L2 =', L2)<jupyter_output>Before changes: L2 = [1, 2, 3, 4, 5]
Single update: L2 = [1, 2, 6, 4, 5]
Multiple update 1: L2 = [1, 22, 33, 44, 5]
Multiple update 2: L2 = [1, 222, 333, 444, 5]<jupyter_text>Deleting and inserting elements<jupyter_code>del L2[2] # deletes element of list L2 with an index of 2
print('Single delete: ', L2)
L2.insert(2, 3) # inserts element 3 to index 2 of L2
print('Single insert: ', L2)
L2.extend([6,7,8]) # extends the list with values from a new list
print('Extend: ', L2)
L2.remove(222) # removes the first occurrence of the element 2 in L2
L2.remove(444)
print('Remove: ', L2) 
L2.insert(1, 2)
L2.insert(3, 4)
print('L2 final: ', L2)
del L # deletes the whole list
print(L)<jupyter_output>Single delete:  [1, 222, 444, 5]
Single insert:  [1, 222, 3, 444, 5]
Extend:  [1, 222, 3, 444, 5, 6, 7, 8]
Remove:  [1, 3, 5, 6, 7, 8]
L2 final:  [1, 2, 3, 4, 5, 6, 7, 8]<jupyter_text>The previous command returned a *NameError*, because we tried to print a list which we had deleted beforehand.We saw a few commands to remove and to insert new elements into a list. The main difference between `del` and `.remove()` is that in ***del*** we have to refer to what we are deleting by **index**, while in ***remove*** we need to refer to the element's **value** instead.When using *insert* we need to know what we are going to put in the list and where are going to put it. For example:```pythonlist.insert(a,b)```adds the value `b` to the list `ls` in the spot designated by the index `a`. All elements after `a` will have their index shifted up by 1.  Using Lists as stacks or queuesA **stack** is a data structure that allows elements to be added and removed only with two operations. The *push* operation adds a new element to the last spot of the stack while the *pop* operation removes the last element in the stack. In a stack we **can't** add or remove elements from any other spot other than the last one! So in order to access the first element we pushed, we first need to pop all other elements that were added after that. This is why the stack is a **LIFO** (Last in, first out) data structure.A **queue** is a similar data structure that has two primary operations. The *enqueue* operation is essentially the same as what called the *push* operation before. It adds a new element to the last spot of the queue. The *dequeue* operation removes one element from the first spot. In a queue the first element that we add is the first element we remove. This is why it is called a **FIFO** (First in, first out) data structure.<jupyter_code>stk = []
stk.append(1) # adds element (1) in the end of list (stk)
stk.append(2) # stk: [1, 2]
stk.append(3) # stk: [1, 2, 3]
stk.pop() # removes the last element from the list.
# pop also returns the last element before removing it!
# to use the list ase a queue we need to pop from the beginning
que = stk[:]
que.pop(0)
# removes firt element. a = 1 (because pop returns removed element)<jupyter_output><empty_output><jupyter_text>We can also pop an item according to it's index e.g:```pythonmy_list.pop(index)```So is we wanted a *stack* we could just use `.append()` and `.pop()`If we want a *queue* we can use `.append()` and `.pop(0)`  Linked listsSometimes in python it's hard to figure out if two variables refer to the same or different lists.<jupyter_code>A = []; B = [] # Independent lists
A = B = [] # Both names point to the same list
A = []
B = A # Both names point to the same list<jupyter_output><empty_output><jupyter_text>These three ways may look similar, but while the first creates **two lists**, the other two create **a single list** with two references to it.Let's look at an example to clarify this:<jupyter_code># First we create a list
A = [1,2,3]
# Now let's make a copy of this list
B = A
# Now let's make some changes to the lists
A.append(4) # we add an element to list A
del B[1] # and we remove one from list B
# Let's see what these lists look like now
print('A =', A, 'B =', B)<jupyter_output>A = [1, 3, 4] B = [1, 3, 4]<jupyter_text>This is not what we expected, right? We wanted to make a copy of the list so that changes in A wouldn't affect B and vice versa.In python if we use the syntax:```python list1 = list2```we don't a copy of the list (as expected), instead we just create a new handle that **refers to the same object** in memory.Let's try the same thing, but create the lists separately this time.<jupyter_code># First we create two lists separately
A = [1,2,3]
B = [1,2,3]
# Now let's make the changes to our lists
A.append(4)
del B[1] 
# Let's see what these lists look like now
print('A =', A, 'B =', B)<jupyter_output>A = [1, 2, 3, 4] B = [1, 3]<jupyter_text>Ok, this works like it's supposed to. But how can we make a **copy** of a list?Turns out there are a few ways to do this. The easiest is through slicing. Remember when we could slice the whole list by typing:```python list[:] ```Though this appears useless at first (Why would we want to slice the whole list?), it is actually really useful, when we realize that it returns a **copy** of our list!Let's try to fool around with our lists again:<jupyter_code>A = [1,2,3]
B = [1,2,3]
print('Initially:')
print('A =', A, 'B =', B)
C = A # C points to the same object as A
D = B[:] # D is a copy of B
print('C =', C, 'D =', D)
# lets make the same changes to the pairs A,C and B,D
C.append(4); D.append(4)
del A[1]; del B[1]
# let's see how the changes affected our lists
print('After changes:')
print('A =', A, 'B =', B)
print('C =', C, 'D =', D)<jupyter_output>Initially:
A = [1, 2, 3] B = [1, 2, 3]
C = [1, 2, 3] D = [1, 2, 3]
After changes:
A = [1, 3, 4] B = [1, 3]
C = [1, 3, 4] D = [1, 2, 3, 4]<jupyter_text>Other built-in functions<jupyter_code>print('len(L1) =', len(L1)) # gives the total length of the list
print('max(L1) =', max(L2)) # returns the element with the max value
print('min(L1) =', min(L2)) # returns the element with the min value
print('sum(L1) =', sum(L2)) # returns the sum of the elements in L2
print("'a's in L1 =", L1.count('a')) # counts how many times 'a' occurs in L1
seq  = ('f','g','h')
L1.extend(seq) # extends contents of seq to L1
print('Extended L1 =', L1)
print("Index of 'd' =", L1.index('d')) # returns the lowest index in list that 'd' appears in L1
print('L2 before reverse: ', L2)
L2.reverse() # reverses order of objects in list
print('L2 after reverse: ', L2)
L2[1] = 0; L2[4] = 6 
print('L2 before sort: ', L2)
L2.sort() # sorts objects in list (default = ascending) 
print('L2 after sort: ', L2)
L2.sort(reverse=True) # descending sort L2
print('L2 after reverse sort: ', L2)
# to get a sorted copy of the list we can use the sorted built in function
L2_cp = sorted(L2)<jupyter_output>len(L1) = 5
max(L1) = 8
min(L1) = 1
sum(L1) = 36
'a's in L1 = 1
Extended L1 = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h']
Index of 'd' = 3
L2 before reverse:  [1, 2, 3, 4, 5, 6, 7, 8]
L2 after reverse:  [8, 7, 6, 5, 4, 3, 2, 1]
L2 before sort:  [8, 0, 6, 5, 6, 3, 2, 1]
L2 after sort:  [0, 1, 2, 3, 5, 6, 6, 8]
L2 after reverse sort:  [8, 6, 6, 5, 3, 2, 1, 0]<jupyter_text>A simple example of using built in functions in practice would be finding the mean of the elements of a list:<jupyter_code>print('list =', L2)
mean = float(sum(L2))/len(L2)
print('mean =', mean)<jupyter_output>list = [8, 6, 6, 5, 3, 2, 1, 0]
mean = 3.875<jupyter_text>IteratorsIterators are built-in objects that allow access to elements in a data structure.Though we won't go into much detail, let's look at the basic syntax<jupyter_code>print('L1 =', L1)
# let's print out the list so that we have an idea what we will be talking about
itr = iter(L1) # declare an iterator object of list L1
print('First element: ', next(itr))  # fetch the first element of L1 and store it in itr
print('Second element: ', next(itr)) # fetch the second element of L1 and store it in itr
# and so on ...<jupyter_output>L1 = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h']
First element:  a
Second element:  b<jupyter_text>Complex lists> Lists can contain more complex items than just the numbers/strings we have seen so far. These complex items can be other lists, tuples, dictionaries or even any other user-created custom object.Let's create a more complex list:<jupyter_code>L3 = ['a_string', 66, (1, 'tuple_string'), ['list_string', [[1.1, 2, 'asdf', False], 2.33, 'list_in_a_list_string', 55]], 'a_second_string']<jupyter_output><empty_output><jupyter_text>We'll first try to figure out it's length:<jupyter_code>print('length: ', len(L3))<jupyter_output>length:  5<jupyter_text>Why does python say that the list has a length of 5, when we clearly added more elements in it?Well, the list L3 **does** have 5 items in it (2 strings, an integer, a tuple and a list).So one of the items in the list is another list. Let's take a look at the inner list (which is the 4th item in L3):<jupyter_code>print('Inner list =', L3[3])

print('Inner list length =', len(L3[3]))<jupyter_output>Inner list = ['list_string', [[1.1, 2, 'asdf', False], 2.33, 'list_in_a_list_string', 55]]
Inner list length = 2<jupyter_text>This list has 2 items: an integer and another list.This even-more-inner list is the 2nd item in the inner list so in order to access it we have to somehow reference the item with an index of 1.We could assign it to a new variable name:<jupyter_code>L3_tmp = L3[3]
# and then reference it
print('Even-more-inner list =', L3_tmp[1])<jupyter_output>Even-more-inner list = [[1.1, 2, 'asdf', False], 2.33, 'list_in_a_list_string', 55]<jupyter_text>Another, easier way to reference this list would be to just add a second set of square brackets after the first.```pythonlst[i][j][k][l]```where:- i: is the index of the outermost list- j: is the index of the second in depth list- k: is the index of the third in depth list- l: is the index of the deepest listIf we use this in our case:<jupyter_code>print('Even-more-inner list =', L3[3][1])
print('length =', len(L3[3][1]))<jupyter_output>Even-more-inner list = [[1.1, 2, 'asdf', False], 2.33, 'list_in_a_list_string', 55]
length = 4<jupyter_text>The first item of this list is again another list<jupyter_code>print('Inner-most list =', L3[3][1][0])<jupyter_output>Inner-most list = [1.1, 2, 'asdf', False]<jupyter_text>Finally we have reached the inner-most list which contains a float, a long integer, a string and a boolean variable.If we wanted to reference a single of these items (lets say the string): ```pythonL3[3][1][0][2]```  Membership conditions> Whit these conditions we can check if a list contains a certain object and return `True` or `False` accordingly.<jupyter_code>print('L1 =', L1)
print('L2 =', L2)
print("'d' in L1: ", 'd' in L1)
print("'d' in L2: ", 'd' in L2)<jupyter_output>L1 = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h']
L2 = [8, 6, 6, 5, 3, 2, 1, 0]
'd' in L1:  True
'd' in L2:  False<jupyter_text>List Operations<jupyter_code>print('L1 + L2 =', L1 + L2) # concatenates lists L1 and L2, i.e. adds L2 at the end of L1
print('L1 * 3 =', L1 * 3) # repeats L1 3 times, i.e. adds L1 to itself 3 times (equivalent to L1+L1+L1)<jupyter_output>L1 + L2 = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 8, 6, 6, 5, 3, 2, 1, 0]
L1 * 3 = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h']<jupyter_text>Other useful functionsNow, we'll create two new lists to illustrate some more advanced python functions.<jupyter_code>a = [1, 2, 3, 4, 5]
b = [2, 2, 9, 0, 9]<jupyter_output><empty_output><jupyter_text>zip> Zip takes two equal-length collections (works on other data structures besides lists) and merges them together in pairs.In python 3 we need to cast the zip object as a list.<jupyter_code>z = list(zip(a,b))
print('zip(a,b) =', z)<jupyter_output>zip(a,b) = [(1, 2), (2, 2), (3, 9), (4, 0), (5, 9)]<jupyter_text>Now we have a single list of pairs (tuples) of elements of the first and the second list.The first element in z is:<jupyter_code>print('z[0] =', z[0])
# is a tuple containing the first element in a and the first element in b<jupyter_output>z[0] = (1, 2)<jupyter_text>The same thing can of course be done with more than two lists.<jupyter_code>c = [11, 22, 33, 44, 55]
z = list(zip(a,b,c))
print('zip(a,b,c) =', z)<jupyter_output>zip(a,b,c) = [(1, 2, 11), (2, 2, 22), (3, 9, 33), (4, 0, 44), (5, 9, 55)]<jupyter_text>map> map takes a function and applies it to each element in an iterable.Let's say we want to perform an elementwise addition of our three lists `a = (a1, a2, ...)` , `b = (b1, b2, ...)` and `c = (c1, c2, ...)`What we would like to to produce is `s = (a1+b1+c1 + a2+b2+c3 + ...)`In python3 we have to cast the map object as a list to get the desired effect.<jupyter_code>s = list(map(sum, zip(a,b,c)))
print('s =', s)<jupyter_output>s = [14, 26, 45, 48, 69]<jupyter_text>What *map* does is it takes a function (in this case: *sum*) and applies it to each element of an iterable (in this case each element is a tuple of the corresponding element of a, b and c (which we produced with *zip*).So `zip(a,b,c)[0]` is the tuple `(a[0], b[0], c[0])`, `zip(a,b,c)[1]` is the tuple `(a[1], b[1], c[1])` and so on.What we want to do is perform the sum function (which produces the sum of such a collection) on each of these tuples nd that is exactly what we accomplish with *map*Likewise if we want to find the minimum element in each index of `a`, `b` and `c`:<jupyter_code>m = list(map(min, zip(a,b,c)))
print('Minimum elements =', m)<jupyter_output>Minimum elements = [1, 2, 3, 0, 5]<jupyter_text>*map* is a nice way to perform elementwise functions to lists.Let's say we want to calculate the square root of the elements in c.Our function is `math.sqrt()` which we want to map across the list `c`.<jupyter_code>from math import sqrt
c_rt = list(map(sqrt, c))
print('Square root of each element =', c_rt)<jupyter_output>Square root of each element = [3.3166247903554, 4.69041575982343, 5.744562646538029, 6.6332495807108, 7.416198487095663]<jupyter_text>Now, let's say we want to square `c_rt` in order to get `c` again.We know of a function `pow(x,2)` which raises `x` to the power of 2, but how do can we apply this to *map*?If the function requires 2 arguments we need to pass two iterables in *map*:<jupyter_code>c_as = list(map(pow, c_rt, [2]*len(c_rt)))
print('c_rt squared =', c_as)<jupyter_output>c_rt squared = [11.0, 22.0, 33.0, 44.0, 55.0]<jupyter_text>*map* requires an iterable object in order to work, so if we want to square each element of `c_rt` in order to produce `c_as` (which is essentially `[ pow(crt[0],2), pow(crt[1],2), ... ]`, we need to create an iterable object (lets say a list) of 2s, i.e `[2, 2, 2, 2, 2]`. ```python[2]*len(c_rt)```creates a list of 2s with the same length as `c_rt` (because the two lists need to be of the same length).By doing this we instruct python to map the function `pow(x,y)` across the lists `c_rt` and `[2, 2, ...]`.Likewise we can define our own functions (we'll learn to do that later) and map them across lists!Let' say we want subtract the values of the elements of *b* from *a*:<jupyter_code>def sub(x,y): return x-y<jupyter_output><empty_output><jupyter_text>Here we defined a sample function called sub which subtracts the number `y` from the number `x`. Don't pay too much attention to the function definition as we'll cover it more extensively in a later tutorial.What matters is that we have a function called `sub` that takes two arguments lets say `x` and `y` and returns the result of the operation `x-y`.<jupyter_code># For example:
sub(3,5)<jupyter_output><empty_output><jupyter_text>Now that we have this function we can just *map* it across our two lists *a* and *b*, to perform an elementwise subtraction of *b* from *a*.<jupyter_code>print('a - b =', list(map(sub, a, b)))<jupyter_output>a - b = [-1, 0, -6, 4, -4]
{'fraction_non_alphanumeric': 0.13947318552207694, 'fraction_numerical': 0.03993746446844798, 'mean_word_length': 4.302436573725195, 'pattern_counts': {'":': 0, '<': 110, '<?xml version=': 0, '>': 116, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '34', 'dir': 'jupyter-structured-clean-dedup', 'id': '475109', 'license': 'MIT', 'n_tokens_mistral': 7834, 'n_tokens_neox': 7266, 'n_words': 3187, 'path': 'notebooks/04_basic_list_operations.ipynb', 'repo_name': 'sniafas/python_ml_tutorial', 'size': '21108'}