<jupyter_start><jupyter_text>Mini Problem**The problem and data is referred from coursera's "Introduction to Data Science in Python" course assignment**In this problem, files of metropolitan regions and associated sports teams is given. The aim is to find out the correlation between win/loss ratio's and population of the city. Here data of NHL(hockey) in 2018 is utilised. The analysis includes data mining, processing and cleaning. Code is developed using python libraries in jupyter platform.The correlation is calculated using [`pearsonr`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.pearsonr.html) function.For cities data follow this link [`wikipedia_data`](https://drive.google.com/file/d/1HyfwbtUOIYS3HK8XGMWxG_v8X7NCrg0G/view?usp=sharing)And for game data follow this [`link`](https://drive.google.com/file/d/1AKfThopH8pxUeYir5Perj1qT9e8LzqPO/view?usp=sharing)<jupyter_code>#importing Necessary libraries
import pandas as pd
import numpy as np
import scipy.stats as stats
import re
#importing datasets
nhl_df=pd.read_csv("nhl.csv")
#
#
cities = pd.read_html("wikipedia_data.html")[1]
cities = cities.iloc[:-1,[0,3,5,6,7,8]]# extraction of necessary data

#small preview of data
cities.head(10)<jupyter_output><empty_output><jupyter_text>Solution formulation Function<jupyter_code># YOUR CODE HERE
#Setting coloumn names
cities.columns = ['Metropolitan area','Population','NFL','MLB','NBA','NHL']
    
#Extracting required data from cities dataframe
df1 = cities[['Metropolitan area','Population','NHL']]
    
# Removing unwanted the contents in 'NFL','MLB','NBA' and 'NHL' columns
df1['NHL'] = df1['NHL'].str.extract('([\w ]*)')
# 
#Modifyig the entries to form good data handling
df1['NHL'].replace({'RangersIslandersDevils':'Rangers,Islanders,Devils','KingsDucks':'Kings,Ducks'}, inplace=True)
    
df1 = df1.assign(NHL = df1.NHL.str.split(",")).explode('NHL')
df1 = df1[df1['NHL'].str.contains("^[\w].*") == True]
#from nhl_df dataframe
df_nhl = nhl_df[nhl_df['year'] == 2018][['team','W','L',]]
df_nhl = df_nhl[df_nhl['W'].str.isnumeric() == True]
    
#modifying the given dta frame to conduct operations
df_nhl['Population'] = None
df_nhl['Metropolitan area'] = None
for team in df1['NHL'].unique():
    idx = df_nhl[df_nhl['team'].str.contains(team)].index[0]
    df_nhl['Population'].loc[idx] = df1[df1['NHL'] == team]['Population'].values[0]
    df_nhl['Metropolitan area'].loc[idx] = df1[df1['NHL'] == team]['Metropolitan area'].values[0]
df_nhl['W/L ratio'] = pd.to_numeric(df_nhl['W'])/(pd.to_numeric(df_nhl['W']) + pd.to_numeric(df_nhl['L']))
df_nhl['Population'] = pd.to_numeric(df_nhl['Population'])
    
#Final dtaframe
final_df = df_nhl.groupby('Metropolitan area').mean()
        
#raise NotImplementedError()
    
population_by_region = final_df['Population'] # pass in metropolitan area population from cities
win_loss_by_region = final_df['W/L ratio'] # pass in win/loss ratio from nhl_df in the same order as cities["Metropolitan area"]

assert len(population_by_region) == len(win_loss_by_region), "Q1: Your lists must be the same length"
assert len(population_by_region) == 28, "Q1: There should be 28 teams being analysed for NHL"
    
ans = stats.pearsonr(population_by_region, win_loss_by_region)[0]

#Display of Answer
print(ans)<jupyter_output>0.012486162921209881
{'fraction_non_alphanumeric': 0.14470517448856798, 'fraction_numerical': 0.022262334536702767, 'mean_word_length': 6.43847874720358, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '33024', 'license': 'MIT', 'n_tokens_mistral': 1219, 'n_tokens_neox': 1150, 'n_words': 272, 'path': 'Main code Data processing.ipynb', 'repo_name': 'ARJUNKVIKRAM/Mini_Problem_Data_Science', 'size': '3324'}