<jupyter_start><jupyter_text>ASSIGNMENT- Replicate the lesson code. - This means that if you haven't followed along already, type out the things that we did in class. Forcing your fingers to hit each key will help you internalize the syntax of what we're doing. - [Lambda Learning Method for DS - By <NAME>](https://docs.google.com/document/d/1ubOw9B3Hfip27hF2ZFnW3a3z9xAgrUDRReOEo-FHCVs/edit?usp=sharing)- Convert the `term` column from string to integer.- Make a column named `loan_status_is_great`. It should contain the integer 1 if `loan_status` is "Current" or "Fully Paid." Else it should contain the integer 0.- Make `last_pymnt_d_month` and `last_pymnt_d_year` columns.<jupyter_code>#I'm running this program local from Anaconda
# I installed wget previously
#I also had to import a file type 'zipfile' to work with the unzip
# some of this lines of code I need some explanation on but I will not look any further at this time
# I need to ask if there is a way to make these lines of code less redundant as I feel I can cut some of it.**
#   df=pd.read_csv("folder_name.zip")** posible solution if only one csv is inside
import wget
lending_url = 'https://resources.lendingclub.com/LoanStats_2018Q4.csv.zip'
df_zip = wget.download(lending_url)
import pandas as pd
import zipfile

zf = zipfile.ZipFile(df_zip)
df = pd.read_csv(zf.open('LoanStats_2018Q4.csv'))
#df_backup_master = df
df.head()
#I'm testing that I did not make any other copies as I had previously made about 3 of them trying out to download 
# with zipfile
df_zip
#making sure that if I need to go back I can easily bring up my backup data frame
# Since I already ran this line I have to be careful not to run it again
#but I have included it into my orignal line of code that unzips
#df_backup_master = df
zf
#this line worked because of built in function to read inside zip without unzipping. 
#but it will not work if there is more than one .csv inside. Beware***
df=pd.read_csv(df_zip, header = 1, na_values = ['n/a'], skipfooter=2, engine='python')
df
# Here I make sure that I dropped footer and aligned header correctly
print(df_backup.shape)
df.shape
df.isnull().sum().sort_values(ascending=False)
#Here I choose to drop a few of the columns based on the fact these 4 have no information in them
#makes the tables easier to read as well
#df_backup2 = df
df = df.drop(columns=['id', 'member_id', 'desc', 'url'])
#looking at this column I can see that I need to drop ' months' and turn it into an int
df['term']
#Here I will def a function and cast to int
def remove_month_to_int(string):
    return int(string.strip(' months'))
#to check it works properly I will make an example
example_cast = '36 months'
remove_month_to_int(example_cast)
df['term'] = df['term'].apply(remove_month_to_int)
df.head()
df['term']
#Make a column named loan_status_is_great. 
#It should contain the integer 1 if loan_status is "Current" or "Fully Paid." 
#Else it should contain the integer 0.
df['loan_status']
print(df['loan_status'].describe(), '\n')
print('Shows us if there are any Null: \n', df['loan_status'].isnull().sum(), '\n')
df['loan_status'].value_counts(dropna=False).head(20)
#find out if there is a way to shorten my test and function with a dictionary.**

#return 1 if string in ["Current" , "Fully Paid"] else 0 **

def assign_loan_status(string):
    if string == 'Current':
        return 1

    elif string == 'Fully Paid':
        return 1
    else:
        return 0
#here I test all three conditions on simple example.
#find out if there is a way to shorten my test and function with a dictionary.**
example_assign = 'Current'
example_assign2 = 'Fully Paid'
example_assign3 = 'Charged Off'

print(assign_loan_status(example_assign), ' for current \n')
print(assign_loan_status(example_assign2), ' for Fully Paid \n')
print(assign_loan_status(example_assign3), ' for every other \n')
#Here I make a new column with the name loan_status_is_great and give it the same values as loan_status
df['loan_status_is_great'] = df['loan_status']
df['loan_status_is_great']
#I'm super happy that at this point I remembered how to use apply without looking back to check hehe
# ask to make sure how I don't get an error if I run this line of code again.**



#df_backup3 = df



df['loan_status_is_great']= df['loan_status_is_great'].apply(assign_loan_status)
df['loan_status_is_great']
#Make last_pymnt_d_month and last_pymnt_d_year columns.
df['last_pymnt_d']
#Here I create the columns I'm going to work with and set them equal to the information I want to get out of them
df['last_pymnt_d_month'] = df['last_pymnt_d']
df['last_pymnt_d_year'] = df['last_pymnt_d']
#here I test to see if I got the right answer
example_workinprogress = 'Aug-2019'
example_workinprogress = example_workinprogress[0:3]
example_workinprogress
df['last_pymnt_d'].isnull().sum()
#I was trying to create this before I realized I should work on this in a different way.
# ask why it was not working.**
#I might have figured it out when I took out the '='.**
import numpy as np


#pass if string is np.NaN else return str[0:3] **

def assign_month(s):
    return s[0:3] if not isinstance(s, float) else s
#    if str is np.NaN:
#        pass
#    else:
#        return str[0:3]

def assign_year(s):
    return s[4:8] if not isinstance(s, int) else s
#    if str is np.NaN:
#        pass
#    else:
#        return str[4:8]
example_date = 'Aug-2018'
print(assign_month(example_date))
print(assign_year(example_date))
df['last_pymnt_d_month'] = df['last_pymnt_d'].apply(assign_month)
df['last_pymnt_d_year'] = df['last_pymnt_d'].apply(assign_year)
print (df[['last_pymnt_d_month', 'last_pymnt_d_year']])
df['last_pymnt_d_month'].isnull().sum()<jupyter_output><empty_output><jupyter_text>STRETCH OPTIONSYou can do more with the LendingClub or Instacart datasets.LendingClub options:- There's one other column in the dataframe with percent signs. Remove them and convert to floats. You'll need to handle missing values.- Modify the `emp_title` column to replace titles with 'Other' if the title is not in the top 20. - Take initiatve and work on your own ideas!Instacart options:- Read [Instacart Market Basket Analysis, Winner's Interview: 2nd place, Kazuki Onodera](http://blog.kaggle.com/2017/09/21/instacart-market-basket-analysis-winners-interview-2nd-place-kazuki-onodera/), especially the **Feature Engineering** section. (Can you choose one feature from his bulleted lists, and try to engineer it with pandas code?)- Read and replicate parts of [Simple Exploration Notebook - Instacart](https://www.kaggle.com/sudalairajkumar/simple-exploration-notebook-instacart). (It's the Python Notebook with the most upvotes for this Kaggle competition.)- Take initiative and work on your own ideas! You can uncomment and run the cells below to re-download and extract the Instacart data<jupyter_code># !wget https://s3.amazonaws.com/instacart-datasets/instacart_online_grocery_shopping_2017_05_01.tar.gz
# !tar --gunzip --extract --verbose --file=instacart_online_grocery_shopping_2017_05_01.tar.gz
# %cd instacart_2017_05_01<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09426518247146681, 'fraction_numerical': 0.01394955615048612, 'mean_word_length': 4.915, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 10, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '18976', 'license': 'MIT', 'n_tokens_mistral': 2387, 'n_tokens_neox': 2272, 'n_words': 883, 'path': 'module4-makefeatures/Jean Fraga LS_DS8_114_Make_Features_Assignment.ipynb', 'repo_name': 'JeanFraga/DS-Unit-1-Sprint-1-Dealing-With-Data', 'size': '7100'}