<jupyter_start><jupyter_text>![Cloud-First](../figures/CloudFirst.png)  SIT742: Modern Data Science **(Module: Python Programming)**---- Materials in this module include resources collected from various open-source online repositories.- You are free to use, change and distribute this package.- If you found any issue/bug for this document, please submit an issue at [tulip-lab/sit742](https://github.com/tulip-lab/sit742/issues)Prepared by **SIT742 Teaching Team**--- Session 2G: Advanced Data Typeswe will learn about advanced data types in addition to the strings and number learnt before. We will examine lists, tuples and dictionaries used for a collection of related data.  Table of Content1 [List](cell_list)2 [Tuple](cell_tuple)3 [Dictionary](cell_dict)   1 List**List is a sequence**Like a string, a *list** is a sequence of values. The values in a list can be any type. We call the values *items** of the list. To create a list, enclose the items in square brackets.For example,<jupyter_code>shoplist = ['apple', 'mango', 'carrot', 'banana']
l = [10, 20, 30, 40]
empty = [ ]   # Initialize an empty list
shoplist
l
empty
shoplist<jupyter_output><empty_output><jupyter_text>The elements of a list do not have to be the same type. An another list can also be nested inside a list. To access the element of a list, use the bracket operator to obtain the value available at the index. Note that the indices of list start at $0$. You can also use negative value as index, if you counts from right. For example, the negative index of last item is $-1$. Try the following examples:<jupyter_code>l = [10, 20, 30, 40]
l[2]
l[-1]
l[-3]<jupyter_output><empty_output><jupyter_text>Here is an example of nested list:<jupyter_code>l = ['apple', 2.0, 5, [10, 20]]
l[1]
l[3]
l[3][1]<jupyter_output><empty_output><jupyter_text>Unlike strings, lists are mutable, which means they can be altered. We use bracket on the left side of an assignment to assign a value to a list item.<jupyter_code>l = [10, 20, 30, 40]
l[1] = 200
l<jupyter_output><empty_output><jupyter_text>**List operation** **In** is used to perform membership operation.  The result of expression equals to **True** if a value exists in the list,  and equals to **False** otherwise.<jupyter_code>shoplist = ['apple', 'mango', 'carrot', 'banana']
'apple' in shoplist
'rice' in shoplist<jupyter_output><empty_output><jupyter_text>Similarly, **In** operator also applies to **string** type. Here are some examples:<jupyter_code>'a' in 'banana'
'seed' in 'banana'    # Test if 'seed' is a substring of 'banana'<jupyter_output><empty_output><jupyter_text>'**+**' is  used for concatenation operation, which  repeats a list a given number of times.<jupyter_code>[10, 20, 30, 40 ] + [50, 60]<jupyter_output><empty_output><jupyter_text>[50, 60]*3      ** List slices **Slicing operation allows to retrieve a slice of of the list. i.e. a part of the sequence. The sliding operation uses square brackets to enclose an optional pair of numbers separated by a colon. Again, to count the position of items from left(first item), start from $0$. If you count the position from right(last item), start from $-1$.<jupyter_code>l = [1, 2, 3, 4]
l[1:3]     # From position 1 to position 3 (excluded)
l[:2]     # From the beginning to position 2 (excluded)
l[-2:]    # From the second right to the beginning<jupyter_output><empty_output><jupyter_text>If you omit both the first and the second indices, the slice is a copy of the whole list.<jupyter_code>l[:]<jupyter_output><empty_output><jupyter_text>Since lists are mutable, above expression is often useful to make a copy before modifying original list.<jupyter_code>l = [1, 2, 3, 4]
l_org = l[:]
l[0] = 8
l
l_org  # the original list is unchanged<jupyter_output><empty_output><jupyter_text>**List methods**The methods most often applied to a list include: - append()- len()- sort()- split()- join() **append()** method adds a new element to the end of a list.<jupyter_code>l= [1, 2, 3, 4]

l
l.append(5)
l.append([6, 7])  #list [6, 7] is nested in list l
l<jupyter_output><empty_output><jupyter_text>**len()** method returns the number of items of a list.<jupyter_code>l = [1, 2, 3, 4, 5]
len(l)
# A list nested in another list is counted as a single item
l = [1, 2, 3, 4, 5, [6, 7]] 
len(l)<jupyter_output><empty_output><jupyter_text>**sort()** arranges the elements of the list from low to high.<jupyter_code>shoplist = ['apple', 'mango', 'carrot', 'banana']
shoplist.sort()
shoplist<jupyter_output><empty_output><jupyter_text>It is worth noted that **sort()** method modifies the list in place, and does not return any value. Please try the following:<jupyter_code>shoplist = ['apple', 'mango', 'carrot', 'banana']
shoplist_sorted = shoplist.sort()
shoplist_sorted    # No value is returned<jupyter_output><empty_output><jupyter_text>There is an alternative way of sorting a list. The build-in function **sorted()** returns a sorted list, and keeps the original one unchanged.<jupyter_code>shoplist = ['apple', 'mango', 'carrot', 'banana']
shoplist_sorted = sorted(shoplist)          #sorted() function return a new list
shoplist_sorted
shoplist<jupyter_output><empty_output><jupyter_text>There are two frequently-used string methods that convert between lists and strings:First, **split()** methods is used to break a string into words:<jupyter_code>s = 'I love apples'
s.split(' ')
s = 'spam-spam-spam'
# A delimiter '-' is specified here. It is used as word boundary
s.split('-')<jupyter_output><empty_output><jupyter_text>Second, **joint()** is the inverse of **split**. It takes a list of strings and concatenates the elements.<jupyter_code>l = ['I', 'love', 'apples']
s = ' '.join(l)
s<jupyter_output><empty_output><jupyter_text>How it works:Since **join** is a string method, you have to invoke it on the *delimiter*. In this case, the delimiter is a space character. So **' '.join()** puts a space between words. The list **l** is passed to **join()** as parameter. For more information on list methods, type "help(list)" in your notebook.  **Traverse a list**The most common way to traverse the items of a list is with a **for** loop. Try the following code:<jupyter_code>shoplist = ['apple', 'mango', 'carrot', 'banana']
for item in shoplist:
    print(item)<jupyter_output><empty_output><jupyter_text>This works well if you only need to read the items of the list. However, you will need to use indices if you want to update the elements. In this case, you need to combine the function **range()** and **len()**.<jupyter_code>l = [2, 3, 5, 7]

for i in range(len(l)):
       l[i] = l[i] * 2

print(l)<jupyter_output><empty_output><jupyter_text>How it works:**len()** returns the number of items in the list, while **range(n)** returns a list from 0 to n - 1. By combining function **len()** and **range()**, **i** gets the index of the next element in each pass through the loop. The assignment statement then uses **i** to perform the operation.   3.2 Tuple**Tuple are immutable **A **tuple** is also a sequence of values, and can be any type. Tuples and lists are very similar. The important difference is that tuples are immutable, which means they can not be changed. Tuples is typically used to group and organizing data into a single compound value.  For example,<jupyter_code>year_born =  ('<NAME>', 1995)
year_born<jupyter_output><empty_output><jupyter_text>To define a tuple, we use a list of values separated by comma. Although it is not necessary, it is common to enclose tuples in parentheses. Most list operators also work on tuples. The bracket operator indexes an item of tuples, and the slice operator works in similar way. Here is how to define a tuple:<jupyter_code>t  = ( ) # Empty tuple
t
t = (1) 
type(t)  # Its type is int, since no comma is following
t = (1,)  # One item tuple; the item needs to be followed by a comma
type(t)<jupyter_output><empty_output><jupyter_text>Here is how to access elements of a tuple:<jupyter_code>t = ('a', 'b', 'c', 'd')
t[0]
t[1:3]<jupyter_output><empty_output><jupyter_text>But if you try to modify the elements of the tuple, you get an error.<jupyter_code>t = ('a', 'b', 'c', 'd')
t[1] = 'B'<jupyter_output><empty_output><jupyter_text>Tuple assignmentTuple assignment allows a tuple of variables on the left of an assignment to be assigned values from  a tuple on the right of the assignment. (We already saw this type of statements in the previous prac)For example,<jupyter_code>t = ('David', '0233', 78)
(name, id, score) = t
name 
id
score<jupyter_output><empty_output><jupyter_text>Naturally, the number of variables on the left and the number of values on the right have to be the same.Otherwise, you will have a system error.<jupyter_code>(a, b, c, d) = (1, 2, 3)<jupyter_output><empty_output><jupyter_text>Lists and tuplesIt is common to have a list of tuples. For loop can be used to traverse the data.  For example,<jupyter_code>t = [('David', 90), ('John', 88), ('James', 70)]

for (name, score) in t:
       print(name, score)<jupyter_output><empty_output><jupyter_text>3 DictionaryA **dictionary** is like an address-book where you can find the address or contact details of a person by knowing only his/her name. The way of achieving this is to associate **keys**(names) with **values**(details). Note that the key in a dictionary must be unique. Otherwise we are not able to locate correct information through the key. Also worth noted is that we can only use immutable objects(strings, tuples) for the keys, but we can use either immutable or mutable objects for the values of the dictionary. This means  we can use either a string, a tuple or a list  for dictionary values.The following example defines a dictionary:<jupyter_code>dict = {'David': 70, 'John': 60, 'Mike': 85}
dict['David']
dict['Anne'] = 92  # add an new item in the dictionary
dict<jupyter_output><empty_output><jupyter_text>**Traverse a dictionary**The key-value pairs in a dictionary are **not** ordered in any manner. The following example uses **for** loop to traversal a dictionary. Notice that the keys are in no particular order.<jupyter_code>dict = {'David': 70, 'John': 60, 'Amy': 85}

for key in dict:
    print(key, dict[key])<jupyter_output><empty_output><jupyter_text>However, we can sort the keys of dictionary before using it if necessary. The following example sorts the keys and stored the result in a list **sortedKey**. The **for** loop then iterates through list **sortedKey**. The items in the dictionary can then be accessed via the names in alphabetical order. Note that dictionary's **keys()** method is used to return a list of all the available keys in the dictionary.<jupyter_code>dict = {'David': 70, 'John': 60, 'Amy': 85}
sortedKeys = sorted(dict.keys())

for key in sortedKeys:
    print(key, dict[key])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1122109158186864, 'fraction_numerical': 0.016003700277520815, 'mean_word_length': 5.20964962665135, 'pattern_counts': {'":': 0, '<': 118, '<?xml version=': 0, '>': 118, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '29', 'dir': 'jupyter-structured-clean-dedup', 'id': '508381', 'license': 'MIT', 'n_tokens_mistral': 3515, 'n_tokens_neox': 3344, 'n_words': 1446, 'path': 'Jupyter/M02-Python/M02G-AdvDataTypes.ipynb', 'repo_name': '1106601034/sit742', 'size': '10816'}