<jupyter_start><jupyter_text>Download census tract data from API  - Available data: https://api.census.gov/data.html  - Variables (aka, fields) you can query for: https://api.census.gov/data/2017/acs/acs5/profile/variables.html  - Browse variables for a place here: https://www.census.gov/acs/www/data/data-tables-and-tools/data-profiles/2014/  - Sample query: https://api.census.gov/data/2017/acs/acs5/profile?get=DP05_0001E&for=tract:400100&in=state:06+county:001  - County FIPS codes: https://www.nrcs.usda.gov/wps/portal/nrcs/detail/?cid=nrcs143_013697   Variables name formatMore info: https://www.census.gov/data/developers/data-sets/acs-5year/data-notes.htmlvariable name format: [TableID]_[RowNumber][VariableType]Example: Variable DP02_0002PE, "Family households (families)", represents the percent estimate for table DP02 row number 2.DP (Data Profile): Table type containing broad social, economic, housing, and demographic information in a total of four profiles.  - DP02: Social Characteristics — includes Education, Marital Status, Relationships, Fertility, Grandparents...   - DP03: Economic Characteristics — includes Income, Employment, Occupation, Commuting to Work...   - DP04: Housing Characteristics — includes Occupancy and Structure, Housing Value and Costs, Utilities...   - DP05: Demographic Characteristics — includes Sex and Age, Race, Hispanic Origin, Housing Units... Variable suffixes:  - E = estimate  - M = margin of error  - PE = percent estimate (of total)  - PM = margin of error for corresponding PE  - A = annotation<jupyter_code>import geopandas as gpd
import getcensus as gc
import os
import pandas as pd
from shapely import geometry
from keys import census_api_key
# which census dataset
dataset = 'acs/acs5'

# which vintage year
year = 2017

# which census variables to retrieve for each tract
variables = {'DP05_0001E':'total_pop',    #total pop
             'DP05_0018E':'median_age',    #median age
             'DP05_0071PE':'pct_hispanic',   #pct pop hispanic or latino
             'DP05_0077PE':'pct_white',   #pct pop non-hispanic white alone
             'DP05_0078PE':'pct_black',   #pct pop non-hispanic black
             'DP05_0080E':'pct_asian',   #pct pop non-hispanic asian
             'DP05_0002PE':'pct_male',   #pct pop male
             'DP04_0007PE':'pct_single_family_home',   #pct single family detached homes
             'DP04_0089E':'med_home_value',    #median value of owner occupied units (dollars)
             'DP04_0037E':'med_rooms_per_home',    #median number of rooms in house
             'DP04_0026PE':'pct_built_before_1940',   #pct structure built 1939 or earlier
             'DP04_0047PE':'pct_renting',   #pct renter-occupied housing units
             'DP04_0005E':'rental_vacancy_rate',    #rental vacancy rate
             'DP04_0049E':'avg_renter_household_size',    #average household size of renter-occupied housing units
             'DP04_0134E':'med_gross_rent',    #median gross rent (dollars)
             'DP03_0062E':'med_household_income',    #median household income
             'DP03_0025E':'mean_commute_time',    #mean travel time to work
             'DP03_0019PE':'pct_commute_drive_alone',   #pct commute drove alone
             'DP03_0128PE':'pct_below_poverty',   #pct people with income below povery level
             'DP02_0057PE':'pct_college_grad_student',   #pct who are students currently enrolled in college or grad school
             'DP02_0079PE':'pct_same_residence_year_ago',   #pct residence 1 year ago was same house
             'DP02_0067PE':'pct_bachelors_degree',   #pct bachelor's degree or higher
             'DP02_0111PE':'pct_english_only',   #pct with english only language spoken at home
             'DP02_0092PE':'pct_foreign_born'}   #pct of population foreign born

# data directories
tracts_path = 'tl_2018_25_tract'
output_path = 'census_tracts_data.geojson'
# download and display census descriptions of each variable
variable_descriptions = gc.get_census_variable_descriptions(dataset=dataset, 
                                                            year=year, 
                                                            variables=variables)
for v, d in variable_descriptions.items():
    print('{}\t{}'.format(variables[v], d['label']))<jupyter_output>total_pop	Estimate!!SEX AND AGE!!Total population
median_age	Estimate!!SEX AND AGE!!Total population!!Median age (years)
pct_hispanic	Percent Estimate!!HISPANIC OR LATINO AND RACE!!Total population!!Hispanic or Latino (of any race)
pct_white	Percent Estimate!!HISPANIC OR LATINO AND RACE!!Total population!!Not Hispanic or Latino!!White alone
pct_black	Percent Estimate!!HISPANIC OR LATINO AND RACE!!Total population!!Not Hispanic or Latino!!Black or African American alone
pct_asian	Estimate!!HISPANIC OR LATINO AND RACE!!Total population!!Not Hispanic or Latino!!Asian alone
pct_male	Percent Estimate!!SEX AND AGE!!Total population!!Male
pct_single_family_home	Percent Estimate!!UNITS IN STRUCTURE!!Total housing units!!1-unit detached
med_home_value	Estimate!!VALUE!!Owner-occupied units!!Median (dollars)
med_rooms_per_home	Estimate!!ROOMS!!Total housing units!!Median rooms
pct_built_before_1940	Percent Estimate!!YEAR STRUCTURE BUILT!!Total housing units!!Built 1939 or earlier
pct_renting	Perc[...]<jupyter_text>Get vars from ACS DP<jupyter_code># load the tracts in our study area
gdf = gpd.read_file(tracts_path).sort_values(by='GEOID')
len(gdf)
%%time
df = gc.get_census_tracts_data(tract_fips=gdf['GEOID'], api_key=census_api_key, dataset=dataset,
                               year=year, variables=variables, clean=True)
# merge the tracts with the acs variables, rename columns, then make sure everything we merged is the same length
merged = pd.merge(left=gdf.set_index('GEOID'), right=df, how='inner', left_index=True, right_index=True)
merged = merged.rename(columns=variables)
assert len(gdf) == len(df) == len(merged)
merged.head()<jupyter_output><empty_output><jupyter_text>Save to disk<jupyter_code>upcast_dispatch = {geometry.Point: geometry.MultiPoint, 
                   geometry.LineString: geometry.MultiLineString, 
                   geometry.Polygon: geometry.MultiPolygon}

def maybe_cast_to_multigeometry(geom):
    caster = upcast_dispatch.get(type(geom), lambda x: x[0])
    return caster([geom])

merged['geometry'] = merged['geometry'].apply(maybe_cast_to_multigeometry)
%%time
merged.reset_index().to_file(output_path, driver='GeoJSON')
print(output_path)<jupyter_output>census_tracts_data.geojson
Wall time: 4.88 s
{'fraction_non_alphanumeric': 0.09717772692601068, 'fraction_numerical': 0.03585049580472922, 'mean_word_length': 4.105919003115265, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 6, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '340872', 'license': 'MIT', 'n_tokens_mistral': 2255, 'n_tokens_neox': 1981, 'n_words': 532, 'path': 'modules/module09 - spatial weights and esda/census/download-census-data.ipynb', 'repo_name': 'JosiahParry/asa', 'size': '6555'}