<jupyter_start><jupyter_text>CL8: Scientific ComputingWelcome to the eighth coding lab!This coding lab focuses on getting comfortable working with data using `pandas`, writing code that adheres to good code style principles, and code testing. **Note!** We have only covered Parts 0-II prior to this week's coding lab. We will only be grading to see concerted effort on those parts. However, we encourage you to revisit this coding lab after testing is discussed in class and give those parts a try.  Part 0: SetupData wrangling often requires additional functionality outside what's included in Python by default. For this, we'll import other functionality from helpful packages. **Import the following packages using their common shortened name found in parentheses:*** `numpy` (`np`)* `pandas` (`pd`)<jupyter_code>### BEGIN SOLUTION
import numpy as np
import pandas as pd
### END SOLUTION
assert pd
assert np<jupyter_output><empty_output><jupyter_text>Part I: `pandas`  `pandas`: dataFiveThirtyEight makes a lot of their data publicly available, so to get some practice using `pandas`, read in a dataframe directly using the `pandas` function `read_csv()`. The input to this function should be the following URL, as a string: https://raw.githubusercontent.com/fivethirtyeight/data/master/voter-registration/new-voter-registrations.csv. Store, this in the object `df`.<jupyter_code>### BEGIN SOLUTION
df = pd.read_csv('https://raw.githubusercontent.com/fivethirtyeight/data/master/voter-registration/new-voter-registrations.csv')
### END SOLUTION
assert isinstance(df, pd.DataFrame) # correct object type
assert df.shape == (106, 4) # correct no. rows & columns<jupyter_output><empty_output><jupyter_text>`pandas`: attributes`pandas` DataFrame objects have a number of attributes. In the cell below, look up the following information specified in the comment of each cell below.Note: each cell should have code that looks like `df.attribute`, where `attribute` is replaced by the attribute that returns the information in the comment<jupyter_code># number of rows and columns

### BEGIN SOLUTION
df.shape
### END SOLUTION
# column names

### BEGIN SOLUTION
df.columns
### END SOLUTION<jupyter_output><empty_output><jupyter_text>`pandas`: methods`pandas` DataFrame objects *also* have a number of helpful methods. In the cell below, look up the following information specified in the comment of each cell below.Each cell should have code that looks like `df.method()`, where `method()` is replaced by the method that returns the information in the comment. Note that some methods will also need information within the parentheses following the method name (i.e. `df.method(arg=val)` and others may require you to specify the series on which you want to operate (i.e. `df['series'].method()`)<jupyter_code># see the first 5 rows of df

### BEGIN SOLUTION
df.head()
### END SOLUTION
# determine which different months are included in df

### BEGIN SOLUTION
df['Month'].unique()
### END SOLUTION
# determine how many different months are included in df

### BEGIN SOLUTION
df['Month'].nunique()
### END SOLUTION
# determine how many times each month shows up in df

### BEGIN SOLUTION
df['Month'].value_counts()
### END SOLUTION
# calculate basic summary statistics
# on New registered voters

### BEGIN SOLUTION
df['New registered voters'].describe()
### END SOLUTION<jupyter_output><empty_output><jupyter_text>Part II: Code StyleBelow a fucntion `get_most_common` is provided. Take a look at the code and examples provided in the cells to understand what the function is doing. Then, consider the code style guidelines discussed in class and edit the code in the cell below to make the code more readable. Note: the name and the functionality of the code will *not* change, but the style (parameter/variable naming, spacing, indentation, capitalization, etc.) shuold be improved.<jupyter_code># edit code style for this function
def get_most_common(d,c):
  MC=d[c].value_counts()
  a=MC.idxmax();b=MC.max()
  return a,b 

### BEGIN SOLUTION
def get_most_common(df, variable):  # better parameter names
    
    # fix indentation; better spacing around operators; improve variable names
    most_common = df[variable].value_counts() 
    max_label = most_common.idxmax()
    max_value = most_common.max()  # separate out onto two lines
    
    return max_label, max_value 
### END SOLUTION
# execute function
get_most_common(df, 'Jurisdiction')
# execute function
# store output in two separate variables
month_common, month_val = get_most_common(df, 'Month')
print(month_common, month_val)<jupyter_output>Jan 24<jupyter_text>Provided is a function you've seen before in A3...but here with particularly terrible code style. As above, edit the function for code style. Its functionality will not change, but its readability will.<jupyter_code># edit code style for this function
def end_chat(i):
    if 'quit' in i:o='Bye';c=False
    else:o=None;c=True      
    return o, c

### BEGIN SOLUTION
def end_chat(input_string):  # better parameter name
    
    # fix indentation; better spacing around operators; improve variable names
    if 'quit' in input_string:
        output = 'Bye'
        chat = False # separate out onto two lines
    else:
        output = None
        chat = True
    
    return output, chat 
### END SOLUTION
# execute function
end_chat('I want to quit')
# execute function
# store output in two separate variables
output, chat = end_chat('I want to quit')
print(output, chat)<jupyter_output>Bye False<jupyter_text>Part III: Code TestingAfter editing the function above, write a test function `test_get_most_common()` that will test the functionality of the function above. Note: you'll likely need to create a dummy dataframe within the function<jupyter_code>### BEGIN SOLUTION
def test_get_most_common():
    # create data frame for testing
    test_df = pd.DataFrame({'col1': ['a', 'b', 'b'], 
                            'col2': [3, 4, 5]}) 
    
    # execute function
    out = get_most_common(test_df, 'col1')
    
    # add assert statements that test function
    assert callable(get_most_common)
    assert isinstance(out, tuple)
    assert out == ('b', 2)
### END SOLUTION
# should pass silently
# when you execute test
test_get_most_common()
# call test function
out = test_get_most_common()
# out should store none if passes silently
assert out == None<jupyter_output><empty_output><jupyter_text>Part IV: `pytest` (*optional*)Let's try out `pytest!`1. Copy the function `get_most_common` to a `functions.py` file (in the same directory as this notebook.)2. Copy the test function `test_get_most_common` to a test file `test_functions.py`. Be sure to add the necessary `import` statement to the top of this file.3. Execute `pytest` in the cell below.<jupyter_code>### BEGIN SOLUTION
!pytest test_functions.py
### END SOLUTION<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08777020165385173, 'fraction_numerical': 0.0030465689830262584, 'mean_word_length': 4.857264231096007, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '139190', 'license': 'MIT', 'n_tokens_mistral': 2044, 'n_tokens_neox': 1929, 'n_words': 819, 'path': '_build/jupyter_execute/labs/CL8-ScientificComputing.ipynb', 'repo_name': 'COGS18/COGS18.github.io', 'size': '6893'}