<jupyter_start><jupyter_text>ONS Boundaries - Choropleth DemoFragments associated with grabbing boundaries data.*Note that a full demo using all local authority ditricts kills MyBinder demo.*  Local Authority Districts (December 2016) Generalised Clipped Boundaries in the UKGet the boundaries for Local Authority areas:http://geoportal.statistics.gov.uk/datasets/7ff28788e1e640de8150fb8f35703f6e_2<jupyter_code>import geopandas

#From the downloads area of the page, grab the link for the shapefile download
url='https://opendata.arcgis.com/datasets/7ff28788e1e640de8150fb8f35703f6e_2.zip?outSR=%7B%22wkid%22%3A27700%2C%22latestWkid%22%3A27700%7D'
gdf = geopandas.read_file(url)

#The .head() method limits the display of the table to the top few rows;
# THings like .head(20) to preview first 20 rows also works
gdf.head()<jupyter_output><empty_output><jupyter_text>We can check the projection:<jupyter_code>gdf.crs<jupyter_output><empty_output><jupyter_text>And preview the shapefile:<jupyter_code>gdf.plot();<jupyter_output><empty_output><jupyter_text>We can easily add a title to the plot:<jupyter_code>#Turn the plot into soemthing we can reference
ax = gdf.plot()

#Add a title
ax.set_title('Local Authorities in the UK')

#Turn off axes
ax.axis('off');<jupyter_output><empty_output><jupyter_text>Interactive MapsPreview shapes from the first few rows using the `folium` package:<jupyter_code>import folium

m = folium.Map(max_zoom=6, location=[53.9, 0.0])
folium.GeoJson(gdf.head()).add_to(m)
m
#You can download a copy of the actual boundary data file if required
# !wget https://opendata.arcgis.com/datasets/52182cdda64d4b15984f6446ca7ee7fd_1.zip?outSR=%7B%22wkid%22%3A27700%2C%22latestWkid%22%3A27700%7D -O wards_fullextent.zip
# !unzip wards_fullextent.zip<jupyter_output><empty_output><jupyter_text>Grab Data to Map AgainstLet's get some data to use as the basis of a choropleth map.We can use something from deprivation indices (we really should check we grabbed boundary files for the correct period...).<jupyter_code>#https://www.gov.uk/government/statistics/english-indices-of-deprivation-2015
#File 10: local authority district summaries

data_url = 'https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/464464/File_10_ID2015_Local_Authority_District_Summaries.xlsx'<jupyter_output><empty_output><jupyter_text>Download the data into a `pandas` dataframe. The orginal file is an Excel spreadsheet with mutliple sheets, so let's load them all and preview the sheetnames:<jupyter_code>import pandas as pd

df = pd.read_excel(data_url, sheet_name=None)

for k in df.keys():
    print(k)<jupyter_output>Notes
IMD
Income
Employment
Education
Health
Crime
Barriers
Living
IDACI
IDAOPI<jupyter_text>We an the preview the data in a single sheet:<jupyter_code>df['Education'].head()<jupyter_output><empty_output><jupyter_text>Generate a Choropleth MapIt's easy enough to combine data from a `pandas` data frame with shape data in a `geopandas` dataframe.The `geopandas` dataframe is used to create a geojson file that the `folium` package can render. Each column name in the `geopandas` dataframe is mapped onto a corresponding `feature.properties.COLUMN_NAME` in the created geojson file. (I'm not sure offhand how column names that include space or punctuation characters are handled: simple column names are easiest.)The data file is also passed in and the key and data columns identified as `columns=[KEYCOL, DATACOL]`.The rendered choropleth map is then coloured accordingly.<jupyter_code>import folium

m =  folium.Map(max_zoom=9, location=[54.5, -0.8])
folium.Choropleth(gdf.head(), key_on='feature.properties.lad16cd',
                  data=df['Education'], 
                  columns=['Local Authority District code (2013)',
                           'Education, Skills and Training - Rank of average rank'],
            fill_color='YlOrBr').add_to(m)
m<jupyter_output><empty_output><jupyter_text>If we try to render the whole of the UK in a MyBinder session, things crash. (I think `geopandas` is quite heavy on resources.)  Exploring the DataWe can exploit the notebook environment further by reating a simple application to explore the data more generally.For example, within the `Education` data sheet, we can explore choropleth maps generated from other columns.We can create tidied up names for the data selection that then refer back to the original column name:<jupyter_code>#We can create a drop down list with values in the list that map onto column names
#A python dict is the data structure that lets us do this

#We use a technique called a dict comprehension to create the dict from a list of column names
#The split separates the column names on '-' elements into two parts
#The parts are refenced by a numercial index value, starting at 0
#Index value 1 is the second item in the split list
#The .strip() command gets rid of leading/trailing whitespace in the string
datacols = {c.split('-')[1].strip():c for c in df['Education'].columns if c.startswith('Education')}
datacols
from ipywidgets import interact

@interact(Indicator=datacols)
def plotEducationChoropleth(Indicator='Education, Skills and Training - Rank of average rank'):
    m = folium.Map(max_zoom=9, location=[54.5, -0.8])
    folium.Choropleth(gdf.head(), key_on='feature.properties.lad16cd',
                      data=df['Education'], 
                      columns=['Local Authority District code (2013)',
                               Indicator],
                fill_color='YlOrBr').add_to(m)
    return m<jupyter_output><empty_output><jupyter_text>Adding Data into GeodataframesWe can merge data into the geodataframe using common columns and then directly plot choropleth maps by selecting the data column we want to plot against.<jupyter_code>#Merge in data
gdf = pd.merge(gdf, df['Education'], 
               how='inner',  #The type of join (what happens if data is in one dataset and not the other)
               left_on='lad16cd', #Column we're merging on in left dataframe
               right_on='Local Authority District code (2013)'#Column we're merging on in right dataframe
              )
gdf.head()<jupyter_output><empty_output><jupyter_text>We can now create a choropleth directly from the dataframe with an inline (non-interactive) plot:<jupyter_code>ax = gdf.plot(column='Education, Skills and Training - Average rank')
ax.axis('off');<jupyter_output><empty_output><jupyter_text>Once again, we can create a simple widget driven app to allow us to explore the data more dynamically. Note also that alternative colour map palettes are available:<jupyter_code>@interact(Indicator=datacols)
def plotEducationChoropleth2(Indicator='Education, Skills and Training - Rank of average rank'):
    ax = gdf.plot(column=Indicator,  cmap='OrRd')
    ax.axis('off');<jupyter_output><empty_output><jupyter_text>Plot a region and it's neighbouring regionsWe can reason about shapes in a limited number of ways. For example, we can tell if one boundary touches another. This means that we can plot a region and the areas that border on it.<jupyter_code>#Via https://gis.stackexchange.com/a/300262/119781

def plotNeighbours(gdf, region='M<NAME>',
                   indicator='Education, Skills and Training - Rank of average rank',
                   cmap='OrRd'):
    ''' Plot choropleth for an indicator relative to a specified region and its neighbours. '''
    
    targetBoundary = gdf[gdf['lad16nm']==region]['geometry'].values[0] 
    neighbours = gdf.apply(lambda row: row['geometry'].touches(targetBoundary) or row['geometry']==targetBoundary ,
                           axis=1)
    
    #Show the data for the selected area and its neighbours
    display(gdf[neighbours][['lad16nm',indicator]].set_index('lad16nm'))
    
    #Generate choropleth
    ax = gdf[neighbours].plot(column=indicator, cmap=cmap)
    ax.axis('off');
plotNeighbours(gdf)
plotNeighbours(gdf, 'Portsmouth')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08877742946708464, 'fraction_numerical': 0.02457680250783699, 'mean_word_length': 4.869021339220015, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 57, 'https://': 5, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '651038', 'license': 'Apache-2.0', 'n_tokens_mistral': 2644, 'n_tokens_neox': 2442, 'n_words': 803, 'path': 'ONS Boundaries-Choropleth Demo.ipynb', 'repo_name': 'ouseful-demos/choropleth-map-demo', 'size': '7981'}