<jupyter_start><jupyter_text>Read in shapefiles for census tracts (`atx_ct`), elementary school catchments (`es_shp`), and 2015-16 [SABS survey](https://nces.ed.gov/programs/edge/SABS) (which lacks Burnet and Austin ISD):<jupyter_code>data_dir = '../../data/'
atx_ct_shp = gpd.read_file(os.path.join(data_dir, '2020_texas_census_tracts/2020_texas_census_tracts.shp'))
es_shp = gpd.read_file(os.path.join(data_dir, 'aisd_shapefiles/20_21_elem_updt_v2.shp'))
sabs_shp = gpd.read_file(os.path.join(data_dir, 'SABS_1516_SchoolLevels/SABS_1516_Primary.shp'))<jupyter_output><empty_output><jupyter_text>Filter out the other 49 states in SABS for the sake of memory and convenience:<jupyter_code>sabs_shp = sabs_shp[sabs_shp['stAbbrev'] == 'TX']
sabs_shp.shape<jupyter_output><empty_output><jupyter_text>Ensure coordinate reference systems (CRS) match. Census tract and AISD shapefiles use NAD83, and SABS is in Open Street Maps (OSM) CRS<jupyter_code>print(atx_ct_shp.crs.name)
print(es_shp.crs.name)
print(sabs_shp.crs.name)<jupyter_output>NAD83
NAD83
WGS 84 / Pseudo-Mercator<jupyter_text>Let's convert to the OSM CRS:<jupyter_code>atx_shp_osm = atx_ct_shp.to_crs({'init': 'EPSG:3857'})
es_shp_osm = es_shp.to_crs({'init': 'EPSG:3857'})
sabs_shp_osm = sabs_shp
assert atx_shp_osm.crs == es_shp_osm.crs == sabs_shp_osm.crs<jupyter_output><empty_output><jupyter_text>Visualize ShapefilesJust to get an idea of what we're looking at.<jupyter_code>sabs_shp_osm.head()
es_shp_osm.head()
atx_shp_osm.head()
es_shp_osm.plot(figsize=(10, 7))<jupyter_output><empty_output><jupyter_text>Calculate the area within each polygon:<jupyter_code>es_shp_osm['es_area'] = es_shp_osm.area
atx_shp_osm['cbg_area'] = atx_shp_osm.area<jupyter_output><empty_output><jupyter_text>Overlay the two polygons:<jupyter_code>es_atx = gpd.overlay(atx_shp_osm, es_shp_osm, how='intersection')<jupyter_output><empty_output><jupyter_text>Calculate overlap area and percent overlap:<jupyter_code>es_atx['overlap_area'] = es_atx.area
# what percent of an elementary catchment corresponds to the underlying CBG?
es_atx['pct_overlap'] = (es_atx['overlap_area'] / es_atx['cbg_area']) * 100<jupyter_output><empty_output><jupyter_text>Grab only the columns of interest to save as a CSV (this will ultimately be further transformed into something like `travel.csv` from the toy contact partitioning model):<jupyter_code>overlaps = es_atx[['GEOID', 'ISD', 'CAMPUS', 'SCHL_YEAR', 'pct_overlap']]<jupyter_output><empty_output><jupyter_text>Add an age level indicator column:<jupyter_code>overlaps['level'] = 'elementary'<jupyter_output><ipython-input-11-e32dde511769>:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  overlaps['level'] = 'elementary'<jupyter_text>Write to file:<jupyter_code>overlaps.to_csv('AISD_elem_census_tract_overlaps.csv')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12182410423452769, 'fraction_numerical': 0.020195439739413682, 'mean_word_length': 8.718354430379748, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '389997', 'license': 'MIT', 'n_tokens_mistral': 1335, 'n_tokens_neox': 1271, 'n_words': 210, 'path': 'workflow/notebooks/20210330_sandbox.ipynb', 'repo_name': 'eho-tacc/aisd_shapefile_integration', 'size': '3070'}