<jupyter_start><jupyter_text>**CST2312 Exercises - 8. Files, 9. Lists, 10. Dictionaries, and 11. Tuples**  **8: Files - Exercises** **Exercise 7.1 (in Lesson 8, Files)** If you are programming in an IDE which interprets code locally on your machine then you can open files using code similar to the code used in examples and exercises in Python for Everybody.  Please understand that if you attempt to read a file without a specified directory path (folder hierarchy) in the file name then that file must be saved in the current working directory where you Python interpreter is running.  See our Supplemental Notebook on Accessing Files for more information.<jupyter_code># This code should work if:
#  1. Your IDE is running Python locally
#  2. The file mbox-short.txt is in the CWD
#  * CWD is the current working directory

fh = open('mbox-short.txt')

for lx in fh:          # process each row of the file
    ly = lx.rstrip()   # create a string without the /n at the end
    print(ly)          # print each line without the extra /n<jupyter_output><empty_output><jupyter_text>The code above^ does not work here in a Colab notebook because Colab is running in the Cloud and not on your local machine.  You see what Colab is doing through a browser on your machine but Colab is doing what it does on Google computers, not on yours.  To approximate the code in Exercise 7.1 in Colab, we will use Google Drive and mount it to Colab so that Colab can access  the My Drive folder and its contents.  That would work as follows: First, we will want Google Drive mounted for access to the My Drive directory.<jupyter_code>from google.colab import drive
drive.mount('/content/drive')<jupyter_output>Mounted at /content/drive<jupyter_text>Then we can open the mbox-short file in the My Drive folder which we mounted to the /content/drive folder in Colab as follows:<jupyter_code># This code should work if:
#  1. Your IDE is Colab and you have Google Drive
#  2. The file mbox-short.txt is in the My Drive folder
#  * My Drive is the root of your Google Drive

fh = open('/content/drive/My Drive/mbox-short.txt')

for lx in fh:          # process each row of the file
    ly = lx.rstrip()   # create a string without the /n at the end
    print(ly)          # print each line without the extra /n<jupyter_output><empty_output><jupyter_text>--- **Exercise 7.2 (in Lesson 8, Files)** Write a program that prompts for a file name, then opens that file and reads through the file, looking for lines of the form:X-DSPAM-Confidence:    0.8475Count these lines and extract the floating point values from each of the lines and compute the average of those values and produce an output as shown below. Do not use the sum() function or a variable named sum in your solution.You can download the sample data at http://www.py4e.com/code3/mbox-short.txt when you are testing below enter mbox-short.txt as the file name. The intended result from is the output: Average spam confidence: 0.7507185185185187 *Initial Code from Py4E Exercise 7.2*<jupyter_code># Use the file name mbox-short.txt as the file name
fname = input("Enter file name: ")
count = 0
tot_fpv = 0
try:
    fh = open(fname)
except:
    print("Unable to open file: ", fname)
    
for line in fh:
    if line.startswith("X-DSPAM-Confidence:"):
        count += 1
    # print(line)
print("Count: ", count)<jupyter_output><empty_output><jupyter_text>The initial code uses try/except to gracefully respond to errors in the file name or the availability of the file.  We can adapt the initial code to work in a Colab environment where the file mbox-short.txt is stored in the root directory of our Google Drive (My Drive folder).  If we do not have it mounted from earlier, we will want Google Drive mounted for access to the My Drive directory.<jupyter_code>from google.colab import drive
drive.mount('/content/drive')<jupyter_output>Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).<jupyter_text>Then we can adapt the code to work with the /content/drive and it's sub-folder My Drive.<jupyter_code>## SETUP - OPEN THE FILE USING MBOX-SHORT.TXT IN GOOGLE DRIVE
file_name = "mbox-short.txt"
# 'borrowing' the following code, though it is overkill
if len(file_name) < 1:
    name = "/content/drive/My Drive/mbox-short.txt"
else:
    name = "/content/drive/My Drive/" + file_name
try:
    exercise_handle = open(name)
except:
    print("Could not open "+name)<jupyter_output><empty_output><jupyter_text>Now we can extract the lines of the original code which were not part of opening the file handle and adjust them. Some differences between opening a file locally and opening a file on Colab but once we have established an open file handle, the rest of our program can work in a similar fashion.<jupyter_code>count = 0     # initialize a counter
line_no = 0
tot_fpv = 0   # initialize a counter
for line in exercise_handle:       # we are using a different handle
    line_no += 1
    if line.startswith("X-DSPAM-Confidence:"):
        count += 1
        # print(line.strip())
print("Count: ", count)<jupyter_output><empty_output><jupyter_text>It works but all we have done so far is count the number of lines which begin with the string we are testing. Let's add more logic to achieve the intended result.<jupyter_code>count = 0     # initialize a counter
tot_fpv = 0   # initialize a counter
for line in exercise_handle:       # we are using a different handle
    if line.startswith("X-DSPAM-Confidence:"):
        count += 1
        colpos = line.find(':')
        number = line[colpos+1:].strip()    # Removes all text except number
        SPAM_C = float(number)
        tot_fpv = tot_fpv + SPAM_C

average = tot_fpv / count
print('Average spam confidence: ', average)<jupyter_output>Average spam confidence:  0.7507185185185187<jupyter_text>Here is a verbose version of the program in exercise 8.5 which uses scaffolding to be able to watch what is happening in the flow of execution.  The following program is written to open a file locally in the current working directory.  See if you can modify it to work in Colabl for you.<jupyter_code>## Read a file and extract float of scores
## from Python for Everybody, Lesson 8 - 
## CST2312 WK06CL07, Professor Patrick 27-Sep-2021
## Using Charles Severance's example ex_08_05


# Use the file name mbox-short.txt as the file name
fname = input("Enter file name: ")
if len(fname) < 1:
    fname = "mbox-short.txt"

# open the file
try:
    local_handle = open(fname)
except:
    print("Unable to open file: ", fname)

# initialize a counter, buckets, and element list
count = 0              # line counter
total_confidence = 0.0 # aggregator for confidence
avg_confidence = 0.0   # total_confidence / count
parsed = list()        # split elements of line

# Some scaffolding to check the variable/object setups
print()
print("I - Initialzed variables/objects now.")
print("File name (fname): ", fname)
print("Counter (count): ", count)
print("Aggregator (total_confidence): ", total_confidence)
print("Avg. confidence (avg_confidence): ", avg_confidence)
print("Parsed line elements (parsed): ", parsed)
go = input("Hit enter to proceed.")
print()


# some scaffolding to articulate flow of control
print()
print("II - Going into the line-by-line processing now.")
# use a for loop to read each line, strip \n, print
for line_read in local_handle:
    line_stripped = line_read.rstrip()
    # Following is a remant from a prior program -- kill it
    # print(line_stripped.upper())
    if not line_stripped.startswith("X-DSPAM-Confidence:"):
        continue
    # scaffolding to peek at each line processed
    print("Rstripped line processed (line_stripped): ", line_stripped)
    count += 1
    parsed = line_stripped.split()
    total_confidence += float(parsed[1])
    # scaffolding to peek at count and parsed list after ea. line
    print("Counter (count): ", count)
    print("Parsed line elements (parsed): ", parsed)
    print("Aggregator (total_confidence): ", total_confidence)
    go = input("Hit enter to proceed.")
    print()

# housekeeping to calculate the average confidence
# using total_confidence / count
avg_confidence = total_confidence / count


# All done -- peek at everything
print()
print("III - Out of the line-by-line processing now.")
print("File name (fname): ", fname)
print("Counter (count): ", count)
print("Aggregator (total_confidence): ", total_confidence)
print("Avg. confidence (avg_confidence): ", avg_confidence)
print("Parsed last-line elements (parsed): ", parsed)
go = input("Hit enter to proceed.")
print()
print("Done")<jupyter_output><empty_output><jupyter_text>---  **9: Lists - Exercises** **Exercise 8.4 (in Lesson 9, Lists)** Open the file romeo.txt and read it line by line. For each line, split the line into a list of words using the split() method. The program should build a list of words. For each word on each line check to see if the word is already in the list and if not append it to the list. When the program completes, sort and print the resulting words in alphabetical order.You can download the sample data at  http://www.py4e.com/code3/romeo.txt  The intended result from the program is the following output: ['Arise', 'But', 'It', 'Juliet', 'Who', 'already', 'and', 'breaks', 'east', 'envious', 'fair', 'grief', 'is', 'kill', 'light', 'moon', 'pale', 'sick', 'soft', 'sun', 'the', 'through', 'what', 'window', 'with', 'yonder'] First, we will want Google Drive mounted for access to the My Drive directory.<jupyter_code>from google.colab import drive
drive.mount('/content/drive')<jupyter_output><empty_output><jupyter_text>Then, we will open "romeo.txt" in the My Drive folder of Google Drive.  No need to prompt the user about this.<jupyter_code>## SETUP - OPEN THE FILE USING ROMEO.TXT IN GOOGLE DRIVE
file_name = "romeo.txt"
# 'borrowing' the following code, though it is overkill
if len(file_name) < 1:
    name = "/content/drive/My Drive/romeo.txt"
else:
    name = "/content/drive/My Drive/" + file_name
try:
    exercise_handle = open(name)
except:
    print("Could not open "+name)<jupyter_output><empty_output><jupyter_text>*Initial Code from Py4E Exercise 8.4*<jupyter_code># fname = input("Enter file name: ")   # no need for this - we hard-coded
# fh = open(fname)  # no need for this because we opened exercise_handle
lst = list()
# for line in fh:    # replacing this with the following line
for line in exercise_handle:    # using our exercise_handle instead of fh
    print(line.rstrip())    # this line need to be indented into the for block<jupyter_output><empty_output><jupyter_text>We cleaned it up a bit to get it to work here in our notebook then indented the print() line to put it in the for block.  So far, we have a program which reads the file, strips off extra /n characters and prints each line of the file.  This has not broken the file down into a sorted list of words though.<jupyter_code>my_list = []                            # initialize an empty list
for line in exercise_handle:
    words = line.split()                # splits a line into a list of words
    # print ("The list words has:", words)  # print the split into words
    # go = input("Type enter to continue")  # pause to look at print
    for word in words:
        if word in my_list:
            continue                    # we do not want duplicates
        my_list.append(word)            # add a new word to the list
print(sorted(my_list))                  # sort in (sort of) alphabetical order<jupyter_output><empty_output><jupyter_text>What if we do not want to count upper case words as different from lower case words?<jupyter_code>my_list = []                            # initialize an empty list
for line in exercise_handle:
    words = line.split()                # splits a line into a list of words
    for word in words:
        word = word.lower()             # remove redundancy from capitalization
        if word in my_list:
            continue                    # we do not want duplicates
        my_list.append(word)            # add a new word to the list
print(sorted(my_list, reverse=True))                  # sort in (better) alphabetical order<jupyter_output><empty_output><jupyter_text>**Exercise 8.5 (in Lesson 9, Lists)** Open the file mbox-short.txt and read it line by line. When you find a line that starts with 'From ' like the following line:From <EMAIL> Sat Jan  5 09:14:16 2008You will parse the From line using split() and print out the second word in the line (i.e. the entire address of the person who sent the message). Then print out a count at the end.Hint: make sure not to include the lines that start with 'From:'. Also look at the last line of the sample output to see how to print the count.You can download the sample data at http://www.py4e.com/code3/mbox-short.txt The intended result from the program should be the output: <EMAIL>zq<EMAIL>zq<EMAIL>gop<EMAIL>aste<EMAIL>There were 27 lines in the file with From as the first word If we do not have it mounted from earlier, we will want Google Drive mounted for access to the My Drive directory.<jupyter_code>from google.colab import drive
drive.mount('/content/drive')<jupyter_output><empty_output><jupyter_text>Then, we will open "mbox-short.txt" in the My Drive folder of Google Drive. No need to prompt the user about this.<jupyter_code>## SETUP - OPEN THE FILE USING MBOX-SHORT.TXT IN GOOGLE DRIVE
file_name = "mbox-short.txt"
# 'borrowing' the following code, though it is overkill
if len(file_name) < 1:
    name = "/content/drive/My Drive/romeo.txt"
else:
    name = "/content/drive/My Drive/" + file_name
try:
    exercise_handle = open(name)
except:
    print("Could not open "+name)<jupyter_output><empty_output><jupyter_text>*Initial code from Py4E Exercise 8.5*<jupyter_code># fname = input("Enter file name: ")   # no need for this - we hard-coded
# if len(fname) < 1:                   # no need for this - we hard-coded
#     fname = "mbox-short.txt"         # no need for this - we hard-coded

# fh = open(fname)      # no need for this because we opened exercise_handle
count = 0


print("There were", count, "lines in the file with From as the first word")<jupyter_output><empty_output><jupyter_text>Well, that^ did not do much.  It is pretty much a 'Hello World' program with an integer value in the print.<jupyter_code>count=0
for line in exercise_handle:
    words = line.split()
    # print(words)
    if len(words) < 3 or words[0] != 'From':
        continue
    # print(words)   # peek at the list words for From lines
    print(words[1])
    count += 1
print('There were %d lines in the file with From as the first word' % count)<jupyter_output><empty_output><jupyter_text>What further functionality could we add to this^ program? --- ---  **10: Dictionaries - Exercises** **Dictionary Exercise - Frequency of Words In a File (Lesson 10)** Here is a slightly revised version of the Python for Everybody exercise which uses a dictionary to count the frequency of words in a text file.  This version of the program will read file from Google My Drive and uses a string for the file path so that only the actual file name needs to be entered.  If the specified file is not in the My Drive directory then the file "romeo.txt" is used. First: if it is not already mounted, mount "My Drive" Google drive so that the file romeo.txt may be referred to with the path and name "/content/drive/My Drive/romeo.txt"<jupyter_code>from google.colab import drive
drive.mount('/content/drive')<jupyter_output><empty_output><jupyter_text>The following two options solve the problem through code to initialize missing keys or using the get method to do the same.<jupyter_code>## SETUP - OPEN THE FILE BASED ON USER SPECIFIED FILE NAME OR ROMEO.TXT
file_name = input("Enter file:")
if len(file_name) < 1:
    name = "/content/drive/My Drive/romeo.txt"
else:
    name = "/content/drive/My Drive/" + file_name
try:
    words_handle = open(name)
except:
    print("Could not open "+name)

## OPTION 1 WHICH INITIALIZES MISSING KEY|VALUE PAIRS THEN COUNTS

words_di = dict()
for line in words_handle :
    line = line.rstrip()
    # print(line)    # scaffolding to peek at line
    words = line.split()
    for word in words :
        # print(words)   # scaffolding to peek at words
        # idiom retrieve/create/update counter
        if word in words_di :
            words_di[word] = words_di[word] + 1
            # print('**Existing**')  # scaffolding to see flow
        else :
            words_di[word] = 1
            # print('**New**')  # scaffolding to see flow

        # print(word, "updated", words_di[word]) # scaffolding

# go = input("Type enter to continue")    # Scaffolding to pause flow
print(words_di)        # scaffolding to peek at word_di
## OPTION 2 WHICH USES THE GET METHOD
words_di = dict()
for line in words_handle :
    line = line.rstrip()
    # print(line)    # scaffolding to peek at line
    words = line.split()
    # print(words)   # scaffolding to peek at words
    for word in words :
        word = word.lower()
        # idiom retrieve/create/update counter
        words_di[word] = words_di.get(word, 0) + 1
        # print(word, "updated", words_di[word]) # Scaffolding

# input("Type enter to continue")    # scaffolding to pause flow
print(words_di)     # scaffolding to peek at word_di<jupyter_output><empty_output><jupyter_text>What if we wanted to know the top-five most frequently used words in the file read?<jupyter_code>## Find the words (keys) of the most frequently used words in the words_di dictionary
print(max(words_di, key=words_di.get))
# Using the max function^ only gives us one result
## Find the words (keys) of the most frequently used words in the words_di dictionary
# We can create a list of the words (keys) which is sorted in descending order of frequency
most_to_least = sorted(words_di, key=words_di.get, reverse=True)
print(most_to_least[:5])
# Printing the first five elements of that list^ only gives us the keys
## Find the words (keys) of the most frequently used words in the words_di dictionary
print(most_to_least)    # scaffolding to look at the most_to_least list (not a dictionary)
## Find the words (keys) of the most frequently used words in the words_di dictionary
# We can use the list of words (keys) by descending frequency to access dictionary pairs
for frequent_word in most_to_least[:5] :
    print(frequent_word, words_di[frequent_word])<jupyter_output><empty_output><jupyter_text>--- *Tip: Here is a routine to find the maximum value in a dictionary and print the associated key and value.*<jupyter_code>a_dictionary = {"a": (9, 1, 5), 99: (2, 5, 7), "c": (9, 9)}

max_key = max(a_dictionary, key=a_dictionary.get) # get key with max value

# print(max_key)    # scaffolding to peek at the integer value max_key
print(max_key, a_dictionary[max_key])<jupyter_output><empty_output><jupyter_text>Same concept but with mixed classes for keys and tuples for values<jupyter_code>a_dictionary = {"a": (9, 1, 5), 99: (2, 5, 7), "c": (9, 9)}

max_key = max(a_dictionary, key=a_dictionary.get) # get key with max value

# print(max_key)    # scaffolding to peek at the integer value max_key
print(max_key, a_dictionary[max_key])<jupyter_output><empty_output><jupyter_text>**Exercise 9.4 (in Lesson 10, Dictionaries)** Write a program to read through the mbox-short.txt and figure out who has sent the greatest number of mail messages. The program looks for 'From ' lines and takes the second word of those lines as the person who sent the mail. The program creates a Python dictionary that maps the sender's mail address to a count of the number of times they appear in the file. After the dictionary is produced, the program reads through the dictionary using a maximum loop to find the most prolific committer. Desired output: <EMAIL> 5 First: mount "My Drive" Google drive so that the file mbox-short.txt may be referred to with the path and name "/content/drive/My Drive/mbox-short.txt"<jupyter_code>from google.colab import drive
drive.mount('/content/drive')<jupyter_output><empty_output><jupyter_text>*Initial Code from Py4E Exercise 9.4*<jupyter_code>name = input("Enter file:")
if len(name) < 1:
    name = "/content/drive/My Drive/mbox-short.txt"
mail_handle = open(name)<jupyter_output><empty_output><jupyter_text>Now, complete the exercise with the required code.  The following two options solve the problem through code to initialize missing keys or using the get method to do the same.<jupyter_code>## STARTUP - USE A FOR LOOP TO PROCESS LINES AS A STRING OF ELEMENTS
from_counts = dict()
for mail_line in mail_handle:
    line_elements = mail_line.split()
    # print(line_elements)   # scaffolding to peek at line_elements
    # input("Type enter to continue")  # scaffolding to pause flow


## OPTION 1 WHICH INITIALIZES MISSING KEY|VALUE PAIRS THEN COUNTS
    if len(line_elements) < 2 or line_elements[0] != 'From':
        continue

    if line_elements[1] not in from_counts:
        from_counts[line_elements[1]] = 1       # First entry
              
    else:
        from_counts[line_elements[1]] += 1     # Additional counts

    # print(from_counts)   # scaffolding to peek at from_counts
    # input("Press enter to continue")  # scaffolding to pause flow



## WRAP-UP - FIND THE MAXIMUM VALUE AND PRINT ITS KEY|VALUE PAIRS
max_key = max(from_counts, key=from_counts.get) # get key with max value
print(max_key, from_counts[max_key])<jupyter_output><empty_output><jupyter_text>Print each key/value pair in the dictionary.<jupyter_code>for key in from_counts:
    print(key, from_counts[key])<jupyter_output><empty_output><jupyter_text>Approach the same problem using the get method<jupyter_code>from_counts = dict()
## STARTUP - USE A FOR LOOP TO PROCESS LINES AS A STRING OF ELEMENTS
for mail_line in mail_handle:
    line_elements = mail_line.split()
    # print(line_elements)   # scaffolding to peek at line_elements
    # input("Type enter to continue")  # scaffolding to pause flow

## OPTION 2 WHICH USES THE GET METHOD
    try: 
        if line_elements[0] == "From" :
            sender = line_elements[1]
            from_counts[sender] = from_counts.get(sender, 0) + 1
    except:
        continue

    # print(from_counts)   # scaffolding to peek at from_counts
    # input("Press enter to continue")  # scaffolding to pause flow


## WRAP-UP - FIND THE MAXIMUM VALUE AND PRINT ITS KEY|VALUE PAIRS
max_key = max(from_counts, key=from_counts.get) # get key with max value
print(max_key, from_counts[max_key])

print(from_counts)  # a little hip-pocket scaffolding to peek at the dictionary<jupyter_output><empty_output><jupyter_text>---  **11. Tuples - Exercises** **Exercise 10.5 (in Lesson 11. Tuples)** The following simple exercise demonstrates the use of the dictionary method items to return a list of tuples which are each comprised of the key and value of each key|value pair.<jupyter_code>d = {'a':10, 'b':1, 'c':22}

for key, val in list(d.items()):
    print(val, key)<jupyter_output><empty_output><jupyter_text>We can print out the contents of a dictionary sorted by the value stored in each key-value pair in descending order.To do this, we first make a list of tuples where each tuple is (value, key). The items method would give us a list of (key, value) tuples, but this time we want to sort by value, not key. Once we have constructed the list with the value-key tuples, it is a simple matter to sort the list in reverse order and print out the new, sorted list.<jupyter_code>my_dictionary = {'a':10, 'b':1, 'c':22}
my_list = list()

# the following for loop processes each key|value pair
# they are added as the tuple (value, key) to my_list
for the_key, the_val in my_dictionary.items() :
    my_list.append( (the_val, the_key) )

print("Printing the list of tuples before sorting")
print(my_list)
print()
my_list.sort(reverse=True)    # sort in descending order
print("Printing the list of tuples after sorting")
print(my_list)
print()<jupyter_output><empty_output><jupyter_text>**Exercise 10.6 (in Lesson 11, Tuples)** First: if it is not already mounted, mount "My Drive" Google drive so that the file romeo-full.txt may be referred to with the path and name "/content/drive/My Drive/romeo-full.txt"<jupyter_code>from google.colab import drive
drive.mount('/content/drive')<jupyter_output><empty_output><jupyter_text>Now we can open a handle to the file romeo-full.txt in /content/drive/My Drive.<jupyter_code>## SETUP - OPEN THE FILE BASED ON USER SPECIFIED FILE NAME OR ROMEO-FULL.TXT
file_name = input("Enter file:")
if len(file_name) < 1:
    name = "/content/drive/My Drive/romeo-full.txt"
else:
    name = "/content/drive/My Drive/" + file_name
try:
    txt_handle = open(name)
except:
    print("Could not open "+name)<jupyter_output>Enter file:<jupyter_text>We can use the following code to process the file and count words using tuples.<jupyter_code># this code uses objects from the string module
import string
# fhand = open('romeo-full.txt')   # we already have txt_handle open
counts = dict()
for line in txt_handle:
    line = line.translate(str.maketrans('', '', string.punctuation))
    line = line.lower()
    words = line.split()
    for word in words:
        if word not in counts:
            counts[word] = 1
        else:
            counts[word] += 1

# Use dictionary items to buld a list of tuples as in exercise 10.5 earlier
lst = list()
for key, val in list(counts.items()):
    lst.append((val, key))

# Sort the dictionary by decreasing value as in exercise 10.5 earlier
lst.sort(reverse=True)

# Use a for loop to print the first ten tuple values
for key, val in lst[:10]:
    print(key, val)<jupyter_output><empty_output><jupyter_text>**Exercise 10.2 (from Lesson 11, Tuples)** *Exercises 10.5, and 10.6 above are examples used in the lesson.  The following exercise 10.2 is the chapter assignment for Lesson 11, Tuples.* Write a program to read through the mbox-short.txt and figure out the distribution by hour of the day for each of the messages. You can pull the hour out from the 'From ' line by finding the time and then splitting the string a second time using a colon.From <EMAIL> Sat Jan  5 09:14:16 2008Once you have accumulated the counts for each hour, print out the counts, sorted by hour as shown below. The intended result is output of the following list of tuples: (04, 3),(06, 1),(07, 1),(09, 2),(10, 3),(11, 6),(14, 1),(15, 2),(16, 4),(17, 2),(18, 1),(19, 1); but each tuple should print unformatted as one integer followed by the other integer then a new line before the next tuple in the list.  *Initial code from Py4E exercise 10.2*<jupyter_code>name = input("Enter file:")
if len(name) < 1:
    name = "mbox-short.txt"
handle = open(name)<jupyter_output><empty_output><jupyter_text>Well, we know that we do not need to start with any of the initial code if we are going to work in Colab with a file from My Drive in Google Drive.  Let's start with mounting Google Drive then opening a handle f or the mbox-short.txt file.<jupyter_code>from google.colab import drive
drive.mount('/content/drive')
name = input("Enter file:")
if len(name) < 1:
    name = "/content/drive/My Drive/mbox-short.txt"
mail_handle = open(name)<jupyter_output><empty_output><jupyter_text>Next, we need to process each desired line ("From" lines) and extract the hour of the day using the split string method.  Then populate a list of the dictionary items as tuples and sort that list for printing.<jupyter_code># initialize a dictionary and a list (for the tuples)
dictionary_hours = dict()
lst = list()

for line in mail_handle:
    words = line.split()
    if len(words) < 2 or words[0] != 'From':
        continue

    col_pos = words[5].find(':')
    hour = words[5][:col_pos]
    if hour not in dictionary_hours:
        dictionary_hours[hour] = 1      # First entry
    else:
        dictionary_hours[hour] += 1     # Additional counts

for key, val in list(dictionary_hours.items()):
    lst.append((key, val))              # Fills list with hour, count of dict

lst.sort()                              # Sorts by hour

for key, val in lst:
    print(key, val)<jupyter_output><empty_output><jupyter_text>---  *review material follows* ---  **10: Dictionaries - Review Material** Lists index their entries based on the position in the listDictionaries are like bags - no orderSo we index the things we put in the dictionary with a “lookup tag” called the "key".  Each key is unique.  Each key is associated with a value, which can be any data type, including lists, or other dictionaries.  Values in a dictionary need not be unique.<jupyter_code>purse = dict()
purse['money'] = 12
purse['candy'] = 3
purse['tissues'] = 75<jupyter_output><empty_output><jupyter_text>After creating an empty dictionary we can add key|value pairs to that dictionary.  Then we can process the dictionary with Python statements and expressions.<jupyter_code>print(purse)
print(purse['candy'])
print("Here are the purse dictionary key|value pairs: ", purse)<jupyter_output><empty_output><jupyter_text>Dictionary literals use curly braces and have a list of key : value pairsYou can make an empty dictionary using empty curly braces<jupyter_code># Dictionary jjj with strings as keys
jjj = { 'chuck' : 1 , 'fred' : 42, 'jan': 100}
print(jjj)
# Empty dictionary ooo
ooo = { }
print(ooo)<jupyter_output><empty_output><jupyter_text>One common use of dictionaries is counting how often we “see” something.  We use the category item as a key and keep an integer counter as the value.<jupyter_code>ccc = dict()
ccc['csev'] = 1
ccc['cwen'] = 1
print(ccc)
# Dictionary values can be used in expressions
ccc['cwen'] = ccc['cwen'] + 1
print(ccc)<jupyter_output><empty_output><jupyter_text>The pattern of checking to see if a key is already in a dictionary and assuming a default value if the key is not there is so common that there is a method called get() that does this for us.  Let's setup a dictionary first, then check for a key with get.<jupyter_code>counts = dict()
names = ['csev', 'cwen', 'csev', 'zqian', 'cwen']
for name in names :
    if name not in counts: 
        counts[name] = 1
    else :
        counts[name] = counts[name] + 1
print(counts)<jupyter_output>{'csev': 2, 'cwen': 2, 'zqian': 1}<jupyter_text>Using the counts dictionary, let's create a function which uses the get statement then we can call the function and print the result in the variable x.<jupyter_code># Create a function using dictionary get to check for a key
if name in counts:
    x = counts[name]
else :
    x = 0
# Call the function with a key as an argument
x = counts.get(name, 0)
# Print the resulting value returned and stored in x
print(x)
print(counts)<jupyter_output><empty_output><jupyter_text>Simplified Counting: We can use get() and provide a default value of zero when the key is not yet in the dictionary - and then just add one to keep a count of instances of each key in a list.  Did we just create a word-count function?<jupyter_code>counts = dict()
names = ['csev', 'cwen', 'csev', 'zqian', 'cwen']
for name in names :
    counts[name] = counts.get(name, 0) + 1
print(counts)<jupyter_output><empty_output><jupyter_text>Let's create a means of counting words entered by the user.  The general pattern to count the words in a line of text is to split the line into words, then loop through the words and use a dictionary to track the count of each word independently.  (Please note that the following example is case-sensitive.)<jupyter_code>counts = dict()
print('Enter a line of text:')
line = input('')

words = line.split()

print('Words:', words)

print('Counting...')
for word in words:
    counts[word] = counts.get(word,0) + 1
print('Counts', counts)<jupyter_output><empty_output><jupyter_text>Definite Loops and Dictionaries: Even though dictionaries are not stored in order, we can write a for loop that goes through all the entries in a dictionary - actually it goes through all of the keys in the dictionary and looks up the values.<jupyter_code>counts = { 'chuck' : 1 , 'fred' : 42, 'jan': 100}
for key in counts:
    print(key, counts[key])<jupyter_output>chuck 1
fred 42
jan 100<jupyter_text>You can get a list of keys, values, or items (both) from a dictionary.  When we have multiple values together as an object, we refer to that object as a tuple.<jupyter_code>jjj = { 'chuck' : 1 , 'fred' : 42, 'jan': 100}
print(list(jjj))

print(list(jjj.keys()))

print(list(jjj.values()))

print(list(jjj.items()))<jupyter_output>['chuck', 'fred', 'jan']
['chuck', 'fred', 'jan']
[1, 42, 100]
[('chuck', 1), ('fred', 42), ('jan', 100)]<jupyter_text>---  **11: Tuples - Review Material** Tuples are like lists. Tuples are another kind of sequence that functions much like a list - they have elements which are indexed starting at 0.<jupyter_code># Create a list and print an element of the list
x = ('Glenn', 'Sally', 'Joseph')
print(x[2])

# Create a tuple
y = ( 1, 9, 2 )
#Print the tuple
print(y)
# Print the max of the tuple
print(max(y))<jupyter_output><empty_output><jupyter_text>Tuples and Assignment: We can also put a tuple on the left-hand side of an assignment statement.  We can even omit the parentheses.<jupyter_code>(x, y) = (4, 'fred')
print(y)

(a, b) = (99, 98)
print(a)<jupyter_output><empty_output><jupyter_text>Tuples and Dictionaries: The items() method in dictionaries returns a list of (key, value) tuples.<jupyter_code>d = dict()
d['csev'] = 2
d['cwen'] = 4
for (k,v) in d.items(): 
    print(k, v)

tups = d.items()
print(tups)<jupyter_output><empty_output><jupyter_text>Tuples are Comparable: The comparison operators work with tuples and other sequences. If the first item is equal, Python goes on to the next element,  and so on, until it finds elements that differ.<jupyter_code>print("Result #1: ", (0, 1, 2) < (5, 1, 2))
print("Result #2: ", (0, 1, 2000000) < (0, 3, 4))
print("Result #3: ", ( 'Jones', 'Sally' ) < ('Jones', 'Sam'))
print("Result #4: ", ( 'Jones', 'Sally') > ('Adams', 'Sam'))<jupyter_output><empty_output><jupyter_text>Sorting Lists of Tuples: We can take advantage of the ability to sort a list of tuples to get a sorted version of a dictionary.  First we sort the dictionary by the key using the items() method and sorted() function.<jupyter_code>d = {'a':10, 'b':1, 'c':22}
d.items()

sorted(d.items())<jupyter_output><empty_output><jupyter_text>Using sorted(): We can do this even more directly using the built-in function sorted that takes a sequence as a parameter and returns a sorted sequence.<jupyter_code>d = {'a':10, 'b':1, 'c':22}
t = sorted(d.items())
print(t)
for k, v in sorted(d.items()):
    print(k, v)<jupyter_output>[('a', 10), ('b', 1), ('c', 22)]
a 10
b 1
c 22<jupyter_text>Sort by Values Instead of Key: If we could construct a list of tuples of the form (value, key) we could sort by value.We do this with a for loop that creates a list of tuples.<jupyter_code>c = {'a':10, 'b':1, 'c':22}
tmp = list()
for k, v in c.items() :
    tmp.append( (v, k) )
print(tmp)

tmp = sorted(tmp, reverse=True)
print(tmp)<jupyter_output>[(10, 'a'), (1, 'b'), (22, 'c')]
[(22, 'c'), (10, 'a'), (1, 'b')]<jupyter_text>Finding the top 10 most common words. First, be sure to have access to the file romeo.txt.<jupyter_code>from google.colab import drive
drive.mount('/content/gdrive')<jupyter_output>Mounted at /content/gdrive<jupyter_text>Then open romeo.txt with the handle fhand and follow the Python for Everybody example.<jupyter_code>fhand = open("/content/gdrive/My Drive/romeo.txt")
# fhand = open('romeo.txt') -->> see the Google Drive open above
counts = {}
for line in fhand:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0 ) + 1

lst = []
for key, val in counts.items():
    newtup = (val, key) 
    lst.append(newtup)

lst = sorted(lst, reverse=True)

for val, key in lst[:10] :
    print(key, val)<jupyter_output><empty_output><jupyter_text>... an even shorter version ...<jupyter_code>c = {'a':10, 'b':1, 'c':22}
print( sorted( [ (v,k) for k,v in c.items() ] ) )<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09758055586313807, 'fraction_numerical': 0.012457092237847415, 'mean_word_length': 4.0701754385964914, 'pattern_counts': {'":': 4, '<': 239, '<?xml version=': 0, '>': 226, 'https://': 0, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '56', 'dir': 'jupyter-structured-clean-dedup', 'id': '466681', 'license': 'MIT', 'n_tokens_mistral': 11584, 'n_tokens_neox': 10967, 'n_words': 4575, 'path': 'CST2312_Exercises_8_9_10_11_py4e.ipynb', 'repo_name': 'ProfessorPatrickSlatraigh/CST2312', 'size': '36645'}