<jupyter_start><jupyter_text>Sport Vouchers Program AnalysisThe goal of this Portfolio task is to explore data from the Federal Government Sport Vouchers program - this is aprogram that provides up to two $100 vouchers for kids to participate in organised sport. Here's the [NSW Active Kids page](https://www.service.nsw.gov.au/transaction/apply-active-kids-voucher), there are similar schemes in other states - this data is from South Australia.This is an exercise in exploring data and communicating the insights you can gain from it.  The source data comesfrom the `data.gov.au` website and provides details of all Sport Vouchers that have been redeemed since February  in SA 2015 as part of the Sport Voucher program:  [Sports Vouchers Data](https://data.gov.au/dataset/ds-sa-14daba50-04ff-46c6-8468-9fa593b9f100/details).  This download is provided for you as `sportsvouchersclaimed.csv`.To augment this data you can also make use of [ABS SEIFA data by LGA](http://stat.data.abs.gov.au/Index.aspx?DataSetCode=ABS_SEIFA_LGA) which shows a few measures of Socioeconomic Advantage and Disadvantage for every Local Government Area. This data is provided for you as `ABS_SEIFA_LGA.csv`. This could enable you to answer questions about whether the voucher program is used equally by parents in low, middle and high socioeconomic areas.   You might be interested in this if you were concerned that this kind of program might just benifit parents who are already advantaged (they might already be paying for sport so this program wouldn't be helping much).Questions:* Describe the distribution of vouchers by: LGA, Sport - which regions/sports stand out? * Are some sports more popular in different parts of the state?* Are any electorates over/under represented in their use of vouchers?* Is there a relationship between any of the SEIFA measures and voucher use in an LGA?A challenge in this task is to display a useful summary of the data given that there are a large number of LGAs and sports involved.  Try to avoid long lists and large tables. Think about what plots and tables communicate the main points of your findings.<jupyter_code>## From Conda I needed to install the packages marked "install"

from pykml.factory import KML_ElementMaker as KML #install
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.cm import viridis
from matplotlib.cm import inferno
from matplotlib.cm import magma
from matplotlib.colors import to_hex
import seaborn as sns
import gmaps #install
import gmaps.datasets 
import json #install
import ipywidgets as widgets

gmaps.configure(api_key='<KEY>')
%matplotlib inline
with open('files/LGA_GDA2020.geojson') as f:
    lgaPos = json.load(f)
# read the sports vouchers data
sa_vouchers = pd.read_csv("files/sportsvouchersclaimed.csv")
#sa_vouchers.head()<jupyter_output><empty_output><jupyter_text>The SEIFA data includes row for each Local Government Area (LGA) but the names of the LGAs have a letter or letters in brackets after the name.  To allow us to match this up with the voucher data we remove this and convert to uppercase. For each LGA the data includes a number of measures all of which could be useful in your exploration.<jupyter_code># read the SEIFA data, create an LGA column by removing the letters in brackets and converting to uppercase
seifa = pd.read_csv('files/ABS_SEIFA_LGA.csv')
lga = seifa["Local Government Areas - 2011"].str.replace(' \([ACSRCDMT]+\)', '').str.upper()
seifa['LGA'] = lga
seifa.head()<jupyter_output><ipython-input-4-8c32cb05a179>:3: FutureWarning: The default value of regex will change from True to False in a future version.
  lga = seifa["Local Government Areas - 2011"].str.replace(' \([ACSRCDMT]+\)', '').str.upper()<jupyter_text>Since there are many rows per LGA we need to use `pivot_table` to create a new data frame with one row per LGA. Hereis an example of doing this to create a table with the different SCORE measures and the population (URP) field.<jupyter_code>LGA_scores = seifa[seifa.MEASURE == 'SCORE'].pivot_table(index="LGA", columns=["INDEX_TYPE"], values="Value")
LGA_scores.head()
LGA_pop = seifa[seifa.MEASURE == 'URP'].pivot_table(index="LGA", columns=["INDEX_TYPE"], values="Value")
LGA_scores['Population'] = LGA_pop.IEO
LGA_scores.head()<jupyter_output><empty_output><jupyter_text>This data frame can then be joined with the vouchers data fram to create one master data frame containing both the voucher data and the SEIFA measures.<jupyter_code>sa_vouchers_scores = sa_vouchers.join(LGA_scores, on='Participant_LGA')
sa_vouchers_scores.head()<jupyter_output><empty_output><jupyter_text>QUESTION 1/2 - Describe the distribution of vouchers  LGAThe main issue in displaying the distribution of vouchers by LGA is the large amount of data to be displayed.There is likely too much data for any traditional graph. As a result of this, I have decided in order to represent the voucher data accurately, two main pieces of information are relevant:1. The overall distribution of vouchers throughout South Australia 2. Whether these vouchers are spread evenly or whether the bulk of the vouchers lie within a few LGAsSo I will create two graphs; one is a heat map of South Australia with brighter colours representing where there are a larger amount of vouchers and darker-lower. The second is a bar chart that displays LGAs that have a voucher count above that of the mean voucher count for the state. This will show us if the bulk of the vouchers lie within a small number of LGAs or if there is a more even distribution.Finally I will display these graphs side by side in order to get an overview of the data.  Cleaning the DataIn order to map the data into the gmaps plugin, I have brought in another file from https://data.sa.gov.au/data/dataset/local-government-areas that contains the positional data of each LGA.  Normalising GPS Data to LGA NamesThis data names the LGAs differently so I first need to clean the positional data so that it can be mapped with the voucher data.<jupyter_code>def changeString(inString):
    inString = inString.replace("CITY OF ", '')
    inString = inString.replace(" CITY COUNCIL", '')
    inString = inString.replace("THE CORPORATION OF THE ", '')
    inString = inString.replace("TOWN OF ", '')
    inString = inString.replace(" COUNCIL", '')
    inString = inString.replace("DC OF ", '')
    inString = inString.replace("THE DC OF ", '')
    inString = inString.replace("THE ", '')
    inString = inString.replace(" DISTRICT", '')
    inString = inString.replace("REGIONAL OF ", '')
    inString = inString.replace(" DC", '')
    inString = inString.replace("UIA ", '')
    inString = inString.replace(' \([ACSRCDMT]+\)', '')
    inString = inString.replace(" REGIONAL", '')
    inString = inString.replace("ORROROO CARRIETON", "ORROROO/CARRIETON")
    inString = inString.replace("COORONG", "THE COORONG")
    inString = inString.replace("TUMBY BAY ", "TUMBY BAY")
    inString = inString.replace("PASTORAL UNINCORPORATED AREA", "NO LGA")
    inString = inString.replace("MUNICIPAL OF ROXBY DOWNS", "ROXBY DOWNS")
    inString = inString.replace("TORRENS ISLAND", "WEST TORRENS")
    inString = inString.replace("RURAL MURRAY BRIDGE", "MURRAY BRIDGE")
    return inString<jupyter_output><empty_output><jupyter_text>Test for names that are in the positional data but not voucher dataI wrote a test that prints out the names of LGAs that are in the positional data but not the voucher data.This allowed me to create the function that would clean the positional data.I left it in here to show my process properly.<jupyter_code>i = 0
while i < len(lgaPos['features']):
    if(changeString(lgaPos['features'][i]['properties']['lga'].upper()) not in changeString(sa_vouchers_scores['Participant_LGA']).values):
        print(changeString(lgaPos['features'][i]['properties']['lga'].upper()))
    
    i = i + 1<jupyter_output><NAME> YANKUNYTJATJARA
RIVERLAND
MARALINGA TJARUTJA<jupyter_text>Displaying Raw Voucher DataWe will first create a map of the voucher data with the gmaps plugin.We will then create a bar graph of the significant values to display next to the map  Creating a Map of the DataThere are two stages to mapping the LGA voucher data. The first is to create an array of colours that correspond to the relative number of vouchers for the LGA (i.e. low numbers of vouchers will get a darker colour and higher numbers will get a brighter colour)The second is to map the location data onto the gmaps plugin along with the corresponding colour for the number of vouchers  Calculating Colours<jupyter_code>def calculate_colour(toCount, category):    
    countMax = sa_vouchers_scores.groupby([category]).count().Participant_ID.max()
    countMin = sa_vouchers_scores.groupby([category]).count().Participant_ID.min()
    countRange = countMax - countMin
    normalisedCount = (toCount - countMin)/countRange
    mpl_color = magma(normalisedCount)
    gmaps_color = to_hex(mpl_color, keep_alpha = False)
    return gmaps_color<jupyter_output><empty_output><jupyter_text>The following loop:1. Looks at each LGA in the positional data 2. Finds the corresponding LGA in the voucher data (if it can't it will add a blank colour to the voucherColours list)3. Passes the count of the LGA vouchers into the calculate_colour function 4. Adds that colour to the voucherColours functionAdditionally it creates a new column in the sa_vouchers_scores dataframe for the colour of each LGA (for use later)<jupyter_code>i = 0
voucherColours = []
while i < len(lgaPos['features']):
    lgaName = changeString(lgaPos['features'][i]['properties']['lga'].upper())
    if((lgaName in changeString(sa_vouchers_scores['Participant_LGA']).values)):
         voucherColours.append(calculate_colour(sa_vouchers_scores[sa_vouchers_scores['Participant_LGA'] == lgaName]['Participant_LGA'].count(), 'Participant_LGA'))
         sa_vouchers_scores.loc[sa_vouchers_scores['Participant_LGA'] == lgaName, "LGA_Count"] = sa_vouchers_scores[["Participant_LGA"]] 
         sa_vouchers_scores.loc[sa_vouchers_scores['Participant_LGA'] == lgaName, "LGA_Colour"] = voucherColours[len(voucherColours) - 1]
    else:
        voucherColours.append((0,0,0,0.3))
        
    i = i + 1<jupyter_output><empty_output><jupyter_text>Creating the mapThis cell uses the gmaps plugin to graph the positions of each LGA along with the colours of the respective count of vouchers.<jupyter_code>fig_layout = {
    'width': '400px',
    'height': '400px',
    'border': '1px solid black',
    'padding': '1px'
}
fig = gmaps.figure(center=(-34.892412, 136.715287), zoom_level = 6, layout=fig_layout)
geojson_layer = gmaps.geojson_layer(
    lgaPos,
   fill_color = voucherColours,  
   fill_opacity=0.8)

fig.add_layer(geojson_layer)


voucherMap = fig<jupyter_output><empty_output><jupyter_text>Creating the table of significant valuesAs the data displayed in the table will need to be in order of voucher count, we first need to create a new colours array for the table to use:<jupyter_code>lgaColours = sa_vouchers_scores[["Participant_LGA", "LGA_Colour"]].sort_values("LGA_Colour")
lgaColours = lgaColours.drop_duplicates(subset=['Participant_LGA'])
lgaColours = lgaColours["LGA_Colour"].to_numpy()<jupyter_output><empty_output><jupyter_text>We then create another column in the original dataframe that represents the number of vouchers in the respective LGA.<jupyter_code>sa_vouchers_scores["LGA_Count"] = sa_vouchers_scores.groupby(["Participant_LGA"]).transform('count')<jupyter_output><empty_output><jupyter_text>We then create a new dataframe that only takes LGAs that have a voucher count above that of the mean voucher count for the dataframe. This will show us if the bulk of the vouchers lie within a small number of LGAs or if there is a more even distribution.<jupyter_code>sa_vouchersHigh = sa_vouchers_scores[sa_vouchers_scores.LGA_Count > sa_vouchers_scores.LGA_Count.mean() ].sort_values("LGA_Count")
firstColour = len(lgaColours) - len(sa_vouchersHigh.groupby("Participant_LGA"))<jupyter_output><empty_output><jupyter_text>We will then create a bar chart that displays the LGAs from the data above<jupyter_code>displayGraph = widgets.Output()
with displayGraph:
    fig = sns.barplot(x= "LGA_Count", y="Participant_LGA", data=sa_vouchersHigh, palette= lgaColours[firstColour: len(lgaColours)])
    fig.set_xlabel("Number of Vouchers")
    fig.set_ylabel("LGA")
    fig.set_title('LGAs Above The Average Number Of Vouchers Claimed', fontsize=15)
    plt.show(fig)<jupyter_output><empty_output><jupyter_text>We will then display this data side by side.<jupyter_code>box_layout = widgets.Layout(display='flex',
                flex_flow='column',
                align_items='center',
                width='100%')
title = widgets.HTML('<h3>Distribution Of Vouchers (raw)</h3>')
widgets.VBox([
    title,
    widgets.HBox([voucherMap, displayGraph], layout={'width': '100%'})
], layout = box_layout)<jupyter_output><empty_output><jupyter_text>SPORTSDisplaying the sport data will be slightly different as it is categorical not numerical data.We will essentially complete the same steps as above except we will map the most common sport for each LGA, and colour these sports based on a colour that makes sense for the sport. We will then graph these sports in a bar chart as above and display the map and graph side by side.  Calculating ColoursThe colours for sport are slightly different as there are specific colours for each sport rather than a range. This means that they need to be hard coded.<jupyter_code>def calculate_sport_colour(sport):
    if sport[0] == "Australian Rules":
        return (218,80,80,0.8)
    if sport[0] == "Netball":
        return (240,249,47,0.8)
    if sport[0] == "Swimming":
        return (60,193,231,0.8)
    if sport[0] == "Basketball":
        return (231,168,60,0.8)
    if sport[0] == "Football (Soccer)":
        return (96,218,70,0.8)
    else:
        return (0,0,0,0.3)
i = 0
sportColours = []
while i < len(lgaPos['features']):
    if(changeString(lgaPos['features'][i]['properties']['lga'].upper()) in changeString(sa_vouchers_scores['Participant_LGA']).values):
         sportColours.append(calculate_sport_colour(sa_vouchers_scores[sa_vouchers_scores['Participant_LGA'] == 
         changeString(lgaPos['features'][i]['properties']['lga'].upper())].groupby(["Participant_LGA"])["Voucher_Sport"].agg(pd.Series.mode).values))
    else:
        sportColours.append((0,0,0,0.3))
    i = i + 1<jupyter_output><empty_output><jupyter_text>Creating Map of Sports<jupyter_code>fig_layout = {
    'width': '400px',
    'height': '400px',
    'border': '1px solid black',
    'padding': '1px'
}
fig = gmaps.figure(center=(-34.892412, 136.715287), zoom_level = 6, layout=fig_layout)
geojson_layer = gmaps.geojson_layer(
    lgaPos,
   fill_color = sportColours,  
   fill_opacity=0.8)

fig.add_layer(geojson_layer)


sportMap = fig<jupyter_output><empty_output><jupyter_text>Creating Graph of Top Sports<jupyter_code>sportList = sa_vouchers_scores.groupby(["Participant_LGA"])["Voucher_Sport"].agg(pd.Series.mode).values
sa_vouchers_scores["Sport_Count"] = sa_vouchers_scores.groupby(["Participant_LGA"]).Voucher_Sport.transform('count')
sportCounts = pd.pivot_table(sa_vouchers_scores, index = "Voucher_Sport", values = "Sport_Count", aggfunc = lambda x: x.value_counts().sum() )
sportCounts = sportCounts[sportCounts.index.isin(sportList)]
sportColoursList = ['#3CC1E7','#F8AD2B','#60DA46','#F0F92F','#DA5050',]
sportGraph = widgets.Output()
with sportGraph:
    fig = sns.barplot(x= "Sport_Count", y=sportCounts.index, data=sportCounts,palette=sportColoursList, order = sportCounts.sort_values('Sport_Count').index)
    fig.set_title("Most Popular Sports in South Australia", fontsize = 15)
    fig.set_ylabel("Sport")
    fig.set_xlabel('Number of Vouchers Claimed')
    plt.show(fig)<jupyter_output><empty_output><jupyter_text>Displaying Map with Graph<jupyter_code>box_layout = widgets.Layout(display='flex',
                flex_flow='column',
                align_items='center',
                width='100%')
title = widgets.HTML('<h3>Distribution Of Sports</h3>')
widgets.VBox([
    title,
    widgets.HBox([sportMap, sportGraph], layout={'width': '100%'})
], layout = box_layout)<jupyter_output><empty_output><jupyter_text>Question 1 SummaryQuestion 1 asked us to describe the distribution of voucher usage as well as the distribution of sports.The voucher usage was extremely concentrated to only a few LGAs. Although there was 69 LGAs in the data, half of the total voucher usage was concentrated in only 7 LGAs. Similarly, although there were a number of sports in the data, when taking the most popular sport in each LGA there were only 5 sports that were significant in the data. These were Swimming, Basketball, Football(Soccer), Netball and Austraian Rules. Australian Rules was by far the most popular sport  Question 2 SummaryAs discussed above, there were 5 sports that were the most popular sport in at least one LGA. Australian Rules was the most popular sport with Netball being the second most popular.   QUESTION 3 - RepresentationAs possible explanation for the voucher distribution being so concentrated within a few LGAs is that the population is also concentrated. In order to find the proportional distribution of data we will need to take population into account.  Normalising Voucher DataIn order to normalise the data, we will create a new dataframe with a Norm_Count value that will be the count of vouchers divided by the population of the LGA multiplyed by 100. This means that Norm_Count will be the percentage of the population of each LGA that has redeemed a voucher.<jupyter_code>normPopDf = pd.pivot_table(sa_vouchers_scores, index = "Participant_LGA", values = ["LGA_Count", "Population", 'IEO','IER','IRSAD', 'IRSD']).reset_index("Participant_LGA")
normPopDf["Norm_Count"] = (normPopDf["LGA_Count"] / normPopDf["Population"]) * 100
normPopDf = normPopDf.dropna()
normPopDf
def calculate_norm_colour(toCount):    
        countMax = normPopDf.Norm_Count.max()
        countMin = normPopDf.Norm_Count.min()
        countRange = countMax - countMin
        normalisedCount = (toCount - countMin)/countRange
        mpl_color = magma(normalisedCount)
        gmaps_color = to_hex(mpl_color, keep_alpha = False)
        return gmaps_color
i = 0
normVCs = []
while i < len(lgaPos['features']):
    lgaName = changeString(lgaPos['features'][i]['properties']['lga'].upper())
    if((lgaName in changeString(normPopDf["Participant_LGA"]).values)):
         normVCs.append(calculate_norm_colour(normPopDf[normPopDf["Participant_LGA"] == lgaName]["Norm_Count"].values[0]))
         normPopDf.loc[normPopDf['Participant_LGA'] == lgaName, "LGA_Colour"] = normVCs[len(normVCs) - 1]
    else:
        normVCs.append((0,0,0,0.3))
        
    i = i + 1
fig_layout = {
    'width': '400px',
    'height': '400px',
    'border': '1px solid black',
    'padding': '1px'
}
fig = gmaps.figure(center=(-34.892412, 136.715287), zoom_level = 6, layout=fig_layout)
geojson_layer = gmaps.geojson_layer(
    lgaPos,
   fill_color = normVCs,  
   fill_opacity=0.8)

fig.add_layer(geojson_layer)


normPopMap = fig
normLgaColours = normPopDf.sort_values("Norm_Count").LGA_Colour.to_numpy()
normPopHigh = normPopDf[normPopDf.Norm_Count > normPopDf.Norm_Count.mean()].sort_values("Norm_Count")
normPopLow = normPopDf[normPopDf.Norm_Count <= normPopDf.Norm_Count.mean()].sort_values("Norm_Count")
normPopGraphHigh = widgets.Output()
plt.figure(figsize=(10, 6))
with normPopGraphHigh:
    fig = sns.barplot(x= "Norm_Count", y="Participant_LGA", data=normPopHigh, palette= lgaColours[40: len(lgaColours)])
    fig.set_xlabel("Percentage Of Population That Have Claimed Vouchers")
    fig.set_ylabel("LGA")
    fig.set_title('LGAs Above The Average Voucher Percentage', fontsize=15)
    plt.xlim(0, 100)
    plt.show(fig)
    
normPopGraphLow = widgets.Output()
plt.figure(figsize=(6, 6.5))
with normPopGraphLow:
    fig = sns.barplot(x= "Norm_Count", y="Participant_LGA", data=normPopLow, palette= lgaColours[0:40])
    fig.set_xlabel("Percentage Of Population That Have Claimed Vouchers")
    fig.set_ylabel("LGA")
    fig.set_title('LGAs Below The Average Voucher Percentage', fontsize=15)
    plt.xlim(0, 100)
    plt.show(fig)
    
box_layout = widgets.Layout(display='flex',
                flex_flow='column',
                align_items='center',
                width='100%',
                height = '100%')
title = widgets.HTML('<h3>Distribution Of Vouchers by Population</h3>')
widgets.VBox([
    title,
    widgets.HBox([normPopMap, normPopGraphLow], layout={'width': '100%'}),
    widgets.HBox([normPopGraphHigh], layout={'width': '100%'}           
    )]
    , layout = box_layout)<jupyter_output><empty_output><jupyter_text>Question 3 SummaryFactoring in the population of each LGA, we get a much more even distribution of vouchers, with a roughly even number of LGAs lying on either side of the mean number of vouchers. However from the data we can see that the LGA of Kingston is far under-represented and the LGA of Karoonda East Murray is far over represented.   QUESTION 4 - Vouchers by SEFIA MeasuresIn comparing the voucher counts to the SEIFA measures for South Australia, the individual LGA scores are not as relevant.As a result of this I will not be using a map to plot the data for this question.Instead, I will produce a scatter plot comparing each of the SEIFA measures to the percentage of the population of each LGA that has claimed a voucher. From this graph, we will be able to see any clear relationship between the SEIFA measures and voucher usage. After this I will calculate the correlations between these measures and give a summary of the relationship of SEIFA measures to voucher usage.<jupyter_code>fig = plt.figure(figsize=(10,6))
sns.scatterplot(data=normPopDf, x = "Norm_Count", y = "IEO" ).set_title("Percentage of LGA Population Claiming Vouchers vs SEIFA Measures",fontsize=15)
sns.scatterplot(data=normPopDf, x = "Norm_Count", y = "IER" ).set_ylabel("SEIFA Score")
sns.scatterplot(data=normPopDf, x = "Norm_Count", y = "IRSAD" ).set_xlabel("Percentage of LGA Population Claiming Vouchers")
sns.scatterplot(data=normPopDf, x = "Norm_Count", y = "IRSD" )
fig.legend(labels=['IEO','IER','IRSAD', 'IRSD'])<jupyter_output><empty_output><jupyter_text>As we can see from this graph, there is no clear linear relationship between any of the SEIFA measues and voucher usage.To be sure of this I will now plot a heat map of the correlation between each of these measures and the voucher usage.<jupyter_code>corr = normPopDf[["Norm_Count","IEO","IER","IRSAD","IRSD"]].corr()[["Norm_Count"]]
corr = corr[corr.index != "Norm_Count"]
corrGraph = sns.heatmap(corr, annot=True, vmax=1)
corrGraph.set_title("Correlation Between Voucher Use and SEIFA Measures",fontsize=15)


#sns.heatmap(df_stats.corr())<jupyter_output><empty_output><jupyter_text>As can be seen from this heatmap, there is somewhat of a relationship between voucher usage and IEO and IER and no relationship at all between IRSAD, IRSD and voucher usage. However, because the correlation between IEO, IER and voucher usage is so low it can likely be said that there is no significant relationship between SEIFA measures and voucher usage in South Australia.   QUESTIONS 1-4 SUMMARY Question 1  LGAsThere was a high concentration of voucher usage within only 7 LGAs.  SportsOnly 5 sports were the most popular in at least one LGA. They were Swimming, Basketball, Football (Soccer), Netball, and Australian Rules. Australian Rules was by far the most popular sport.  Question 2As discussed above Australian Rules is popular in most parts of the state, with Netball being the second most popular.  Question 3Factoring in population, voucher usage was much more evenly spread. However, Kingston was under represented and Karoonda East Murray was over represented.  Question 4There likely wasn't a significant relationship between SEIFA measures and voucher usage.  Challenge - Queensland_Note: this is an extra task that you might take on to get a better grade for your portfolio.  You can get a good pass grade without doing this._ Queensland has a similar program called [Get Started](https://data.gov.au/dataset/ds-qld-3118838a-d425-48fa-bfc9-bc615ddae44e/details?q=get%20started%20vouchers) and we can retrieve data from their program in a similar format.  The file [round1-redeemed_get_started_vouchers.csv](files/round1-redeemed_get_started_vouchers.csv) contains records of the vouchers issued in Queensland. The date of this data is not included but the program started in 2015 so it is probably from around then.  The data includes the LGA of the individual but the name of the activity is slightly different.  To do a comparable analysis you would need to map the activity names onto those from South Australia.<jupyter_code>qld_vouchers = pd.read_csv('files/round1-redeemed_get_started_vouchers.csv')
qld_vouchers.head()
# Join the QLD data with the LGA data as before to get population and SIEFA data integrated
qld_vouchers['LGA'] = qld_vouchers['Club Local Government Area Name'].str.replace(' \([RC]+\)', '').str.upper()
qld_vouchers_scores = qld_vouchers.join(LGA_scores, on='LGA')
qld_vouchers_scores.head()<jupyter_output><ipython-input-39-d688531e2bfc>:2: FutureWarning: The default value of regex will change from True to False in a future version.
  qld_vouchers['LGA'] = qld_vouchers['Club Local Government Area Name'].str.replace(' \([RC]+\)', '').str.upper()
{'fraction_non_alphanumeric': 0.09435732999648067, 'fraction_numerical': 0.014507488366636687, 'mean_word_length': 5.090497737556561, 'pattern_counts': {'":': 5, '<': 101, '<?xml version=': 0, '>': 98, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '181994', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 8346, 'n_tokens_neox': 7820, 'n_words': 2677, 'path': 'Portfolio 2.ipynb', 'repo_name': 'liamflannery/data-science-portfolio-liamflannery', 'size': '25622'}