<jupyter_start><jupyter_text>IST 652 Lab 5 Instructions- Complete all 6 questions in this assignment.- You may work with others,  but the work you submit must be your own . You can differentiate your work by adding comments or changing the values you use to test your code. However, submitting some else's work as your own is an academic integrity violation and will be raised to academic affairs.- It is always better to attempt a problem as partial credit may be granted. Submission Guide:- Submit your answers on BlackBoard by Saturday 2019-03-30.- The file must be either a .py or .ipynb file type.- The name of the file you submit should be  ist652_lab5_lastname.py (.ipynb) . Grading [ 6 total points ]For Each Questions (1-6), the following credit will be awarded:- 0.75 for printing the correct answer to the console.- 0.15 for approaching the problem efficiently.- 0.05 for properly documenting and commenting your code. --- Questions  Analysis of Given Names @ Data.Gov- https://catalog.data.gov/dataset/most-popular-baby-names-by-sex-and-mothers-ethnic-group-new-york-city-8c742![image.png](attachment:image.png) ---  ( 1a ) Fetch the online data from the following location and convert it into a python JSON object:- https://data.cityofnewyork.us/api/views/25th-nujf/rows.json?accessType=DOWNLOAD- please use the packages below which we covered in class (urllib and json)  (1b) Create a new list variable called "name_data" which are the values associated with the JSON key = 'data'. [1 point]<jupyter_code>import urllib
import json
# Enter your code here, printing relevant answers to console:<jupyter_output><empty_output><jupyter_text>--- ( 2a ): Create a dataframe from the values in "name_data" list- For each name in the "name_data" list, the only data we care about are the following indices:    - name[8] which is the year    - name[9] which is the gender    - name[10] which is the ethnicity    - name[11] which is the name    - name[12] which is the instance of the name    - name[13] which is the rank of the name within the year/gender/enthnicity group- Additionally, be sure that series which are numbers are cast as float or np.float datatypes. ( 2b ): Display the summary statistics of both the numeric and string data in the dataframe using the describe method. [1 point]<jupyter_code># Enter your code here, printing relevant answers to console:
import pandas as pd<jupyter_output><empty_output><jupyter_text>----  ( 3a ):  Create a new dataframe by aggregating the dataframe from (2a) to the following level:- Group by Gender, Name and return the sum of name instances (name[12] from 2a) ( 3b ): Count the number of distinct names in the dataframe created in (3a).- If Alex is a FEMALE and a MALE name, still only count this name once. ( 3c ): Create a list of distinct names which are assigned to both FEMALE and MALE children and print the length of the list. [1 point]<jupyter_code># Enter your code here, printing relevant answers to console:<jupyter_output><empty_output><jupyter_text>----  ( 4 ) Create a new pivoted dataframe ( e.g. use pd.pivot() ) from (2a) such that:- We are only considering:    - year = '2013'    - gender = 'FEMALE'    - rank <= 10- The pd.pivot() method maps the following parameters:    - index = 'ethnicity'    - columns = 'rank'    - values = 'name'The resulting dataframe should look like:![image.png](attachment:image.png) [1 point]<jupyter_code># Enter your code here, printing relevant answers to console:<jupyter_output><empty_output><jupyter_text>---  MongoDB --- ( 5a ): Convert the dataframe from (2b) to a list of dictionaries.- It's highly recommended that you use the method pd.to_dict(orient='records') ( 5b ): Create a MongoDB database called 'names_db'- IMPORTANT: Don't forget to make sure the MongoDB session is active    - run ```$ mongod``` in the terminal- Use pymongo package for operating on MongoDB databases ( 5c ): Load the list of dictionaries from (5a) into a collection called "names" in the database "names_db" from (5b) [1 point]<jupyter_code># Enter your code here, printing relevant answers to console:
from pymongo import MongoClient
client = MongoClient('localhost', 27017)<jupyter_output><empty_output><jupyter_text>--- ( 6a ): Return the following records from the MongoDB collection "names" you created in (5b)- All records which are of rank >= 100- All records where the name == 'Grace'- All records where the gender == 'MALE' AND where the year = '2016'    - Also, return the records sorted by gender  ( 6b ): Count the number of records in the Mongo Collection "names" where:- The name "Sarah" is in the top 5 (i.e. rank <= 5)- The rank >= 15 and the year is 2012- The count of name instances is greater than 350 [1 point]<jupyter_code># Enter your code here, printing relevant answers to console:<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08550724637681159, 'fraction_numerical': 0.02132505175983437, 'mean_word_length': 4.279781420765027, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '311557', 'license': 'CC0-1.0', 'n_tokens_mistral': 1423, 'n_tokens_neox': 1329, 'n_words': 661, 'path': 'IST652 - Lab 5.ipynb__403613_1_1553705600000.ipynb', 'repo_name': 'Pammu6/IST652-Scripting-for-Data-Analysis', 'size': '4830'}