<jupyter_start><jupyter_text>Data Manipulation with Python  Learning Goals- Construct list and dictionary comprehensions- Extract data from nested data structures- Write functions to transform data  Lists  List MethodsMake sure you're comfortable with the following list methods:- `.append()`: adds the input element to the end of a list- `.pop()`: removes and returns the element with input index from the list- `.extend()`: adds the elements in the input iterable to the end of a list- `.index()`: returns the first place in a list where the argument is found- `.remove()`: removes element by value- `.count()`: returns the number of occurrences of the input element in a listQuestion: What's the difference between `.remove()` and `del`?            Answer here        .remove() removes an element by value;    del removes an element by position  List Comprehension List comprehension is a handy way of generating a new list from existing iterables.Suppose I start with a simple list.<jupyter_code>primes = [2, 3, 5, 7, 11, 13, 17, 19]<jupyter_output><empty_output><jupyter_text>What I want now to do is to build a new list that comprises doubles of primes. I can do this with list comprehension! The syntax is: `[ f(x) for x in  if ]`<jupyter_code>prime_doubles = [x*2 for x in primes]
prime_triples = [x*3 for x in primes]
prime_doubles<jupyter_output><empty_output><jupyter_text>Aside: List Comprehensions Vs. `for`-Loops Yes, I could do the same work with `for`-loops:<jupyter_code>prime_doubles2 = []
for prime in primes:
    prime_doubles2.append(prime*2)
prime_doubles2<jupyter_output><empty_output><jupyter_text>But list comprehensions are more efficient: The syntax is simpler, and they're also faster. Also, you'll see them in other people's code, so you'll have to know how to work with them!  End of Aside I can use list comprehension to build a list from objects other than lists:<jupyter_code>my_dict = dict(zip(range(5), 'aeiou'))
my_dict
[v for k, v in my_dict.items() if k % 4 == 0]
names = ['<NAME>', '<NAME>', '<NAME>',
        '<NAME>', '<NAME>', '<NAME>']

splits = [name.split() for name in names]

[name1[0]+'. '+name2[0]+'.' for (name1, name2) in splits]<jupyter_output><empty_output><jupyter_text>Exercises 1. Use a list comprehension to extract the odd numbers from this set:<jupyter_code>nums = set(range(1000))
odds = [x for x in nums if x % 2 == 1]
len(odds)<jupyter_output><empty_output><jupyter_text>Answer        [num for num in nums if num % 2 == 1]     2. Use a list comprehension to take the first character of each string from the following list of words.<jupyter_code>words = ['carbon', 'osmium', 'mercury', 'potassium', 'rhenium', 'einsteinium',
        'hydrogen', 'erbium', 'nitrogen', 'sulfur', 'iodine', 'oxygen', 'niobium']
initials = [word[0].upper() for word in words]
initials<jupyter_output><empty_output><jupyter_text>Answer        [word[0] for word in words]     3. Use a list comprehension to build a list of all the names that start with 'R' from the following list. Add a '?' to the end of each name.<jupyter_code>names = ['Randy', 'Robert', 'Alex', 'Ranjit', 'Charlie', 'Richard', 'Ravdeep',
        'Vimal', 'Wu', 'Nelson']
r_names = [name + "?" for name in names if name[0] == "R"]
r_names<jupyter_output><empty_output><jupyter_text>Answer        [name+'?' for name in names if name[0] == 'R']      Dictionaries  Dictionary MethodsMake sure you're comfortable with the following dictionary methods:- `.keys()`: returns an array of the dictionary's keys- `.values()`: returns an array of the dictionary's values- `.items()`: returns an array of key-value tuples  Dictionary Comprehension Much like list comprehension, I can use dictionary comprehension to build dictionaries from existing iterables.<jupyter_code>my_dict = {'who': 'flatiron school', 'what': 'data science',
           'when': 'now', 'where': 'here', 'why': '$',
           'how': 'python'}<jupyter_output><empty_output><jupyter_text>Remember that the `.items()` method will return a collection of doubles:<jupyter_code>my_dict.items()<jupyter_output><empty_output><jupyter_text>So I can use a pair of variables to range over it:<jupyter_code>{k: v + '!' for k, v in my_dict.items() if k.startswith('w')}<jupyter_output><empty_output><jupyter_text>The same thing works for any collections of doubles:<jupyter_code>{k**2: v**2 for k, v in [(0, 1), (2, 3), (4, 5)]}<jupyter_output><empty_output><jupyter_text>`zip`Remember that `zip` is a handy way of pairing up two or more iterables:<jupyter_code>dict(zip(range(5), ['apple', 'orange', 'banana', 'lime', 'blueberry']))
tuple(zip(range(1, 5), 'a'*4, 'b'*4, 'c'*4, 'd'*4, 'e'*4))<jupyter_output><empty_output><jupyter_text>Dictionary Comprehension Using `zip`<jupyter_code>{k: v for k, v in zip(range(5), range(0, 10, 2))}
scores = [.858, .873, .868]
{'model' + str(j+1): scores[j] for j in range(3)}<jupyter_output><empty_output><jupyter_text>Exercises 1. Use a dictionary comprehension to pair up the countries in the first list with their corresponding capitals in the second list:<jupyter_code>list1 = ['USA', 'France', 'Canada', 'Thailand']
list2 = ['Washington', 'Paris', 'Ottawa', 'Bangkok']

capitals = {k: v for k, v in zip(list1, list2)}
capitals<jupyter_output><empty_output><jupyter_text>Answer        {country: capital for (country, capital) in zip(list1, list2)}     2. Use a dictionary comprehension to make each of the characters in the following list a key with the value 'fictional character'.<jupyter_code>chars = ['Pinocchio', 'Gilgamesh', '<NAME>', '<NAME>']
characters = {name: 'fictional character' for name in chars}
characters<jupyter_output><empty_output><jupyter_text>Answer    {char: 'fictional character' for char in chars}      Nesting Just as we can put lists and dictionaries inside of other lists and dictionaries, we can also put comprehensions inside of other comprehensions.<jupyter_code>lists = [['morning', 'afternoon', 'night'], ['read', 'code', 'sleep']]
[[item[0] for item in small_list] for small_list in lists]<jupyter_output><empty_output><jupyter_text>Nested Structures It will be well worth your while to practice accessing data in complex structures. Consider the following:<jupyter_code>customers = {
    'bill': {'purchases': {'movies': ['Terminator', 'Elf'],
                     'books': []}, 'id': 1},
            'dolph': {'purchases': {'movies': ['It Happened One Night'],
                     'books': ['The Far Side Gallery']}, 'id': 2},
            'pat': {'purchases': {'movies': [],
                   'books': ['Seinfeld and Philosophy', 'I Am a Bunny']},
                   'id': 3}
}<jupyter_output><empty_output><jupyter_text>**Q**: How would we access 'I Am a Bunny'?**A**: The outermost "layer" has a name: 'customers', and that object is a dictionary:`customers`The key we are interested in is 'pat', since that's where 'I Am a Bunny' is located:`customers['pat']`The value corresponding to the key 'pat' is also a dictionary, and in this "lower-down" dictionary, the key we are interested in is 'purchases':`customers['pat']['purchases']`The value corresponding to the key 'purchases' is yet another dictionary, and here the key of interest is `books`:`customers['pat']['purchases']['books']`The value corresponding to the key 'books' is a list, and 'I Am a Bunny' is the second element in that list:`customers['pat']['purchases']['books'][1]`<jupyter_code>customers['pat']['purchases']['books'][1]<jupyter_output><empty_output><jupyter_text>Exercises 1. From the list below, make a list of dictionaries where the key is the person's name and the value is the person's home phone number.<jupyter_code>phone_nos = [{'name': 'greg', 'nums': {'home': 1234567, 'work': 7654321}},
          {'name': 'max', 'nums': {'home': 9876543, 'work': 1010001}},
            {'name': 'erin', 'nums': {'home': 3333333, 'work': 4444444}},
            {'name': 'joél', 'nums': {'home': 2222222, 'work': 5555555}},
            {'name': 'sean', 'nums': {'home': 9999999, 'work': 8888888}}]

home_phones = [{item['name'].title(): 
                str(item['nums']['home'])[:3] + '-' +
                str(item['nums']['home'])[4:7]} 
               for item in phone_nos]
home_phones<jupyter_output><empty_output><jupyter_text>Answer    [{item['name']: item['nums']['home']} for item in phone_nos]     2. From the customers dictionary above, build a dictionary where the customers' names are the keys and the movies they've bought are the values.<jupyter_code>movie_purchases = {customer.title(): customers[customer]['purchases']['movies'] for customer in customers.keys()}
movie_purchases<jupyter_output><empty_output><jupyter_text>Answer    {customer: customers[customer]['purchases']['movies'] for customer in customers.keys()}      FunctionsThis aspect of Python is _incredibly_ useful! Writing your own functions can save you a TON of work - by _automating_ it. Creating FunctionsThe first line will read:'def' + _your function's name_ + '( )' + ':'Any arguments to the function will go in the parentheses.Let's try building a function that will automate the task of finding how many times a given number can be evenly divided by 2.<jupyter_code>def divide_by_two_repeatedly(num):
    counter = 0
    while num % 2 == 0:
        num /=2
        counter += 1
    return counter<jupyter_output><empty_output><jupyter_text>Calling FunctionsTo _call_ a function, simply type its name, along with any necessary arguments in parentheses.<jupyter_code>divide_by_two_repeatedly(64)<jupyter_output><empty_output><jupyter_text>Default Argument ValuesSometimes we'll want the argument(s) of our function to have default values.<jupyter_code>def cheers(person='aaron', job='data scientist', age=30):
    return f'Hooray for {person.title()}. You\'re a {job} and you\'re {str(age)}!'
cheers('greg', 'scientist', 130)
cheers('cristian', 'git enthusiast', 93)
cheers()<jupyter_output><empty_output><jupyter_text>Exercises 1. Build a function that will return $2^n$ for an input $n$.<jupyter_code>def two_to_the_power_of(power):
    return f'Two to the power of {str(power)} is {str(2 ** power)}!'

two_to_the_power_of(0.5)<jupyter_output><empty_output><jupyter_text>Answer    def expo(n):    return 2**n     2. Build a function that will take in a list of phone numbers as strings and return the same as integers, removing any parentheses ('(' and ')'), hyphens ('-'), and spaces.<jupyter_code>def tidy_phone_nos(list_of_phone_numbers):<jupyter_output><empty_output><jupyter_text>Answer    def int_phone(string):    return int(string.replace('(', '').replace(')', '').replace('-', '').replace(' ', ''))     3. Build a function that returns the mode of a list of numbers.     Answer        def mode(lst):    counts = {num: lst.count(num) for num in lst}    return [num for num in counts.keys() if counts[num] == max(counts.values())]<jupyter_code>def get_mode(list_of_numbers):      
    frequency_dict = {}
    
    for item in data:
        if (item in frequency_dict)==True:
            frequency_dict[item]+=1
        else:
            frequency_dict[item]=1

    mode=[]

    for element in frequency_dict.items():
        if element[1] == max(list(frequency_dict.values())):
            mode.append(element[0])
    
    
    return mode<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13612983327421072, 'fraction_numerical': 0.016849946789641718, 'mean_word_length': 4.846034214618974, 'pattern_counts': {'":': 0, '<': 113, '<?xml version=': 0, '>': 113, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 6, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '26', 'dir': 'jupyter-structured-clean-dedup', 'id': '18035', 'license': 'MIT', 'n_tokens_mistral': 3777, 'n_tokens_neox': 3531, 'n_words': 1233, 'path': 'Phase_1/ds-python_data_manipulation-main/python_data_manipulation.ipynb', 'repo_name': 'clareadunne/ds-east-042621-lectures', 'size': '11324'}