<jupyter_start><jupyter_text>Inner-zonal trips Preperation of calibration dataAssignment of inner-zonal trips is not possible in a classic transport model. Trip lengths by mode and purpose must be approximated with MiD2017 data using a logistic regression<jupyter_code>input_path = '../input_static/'
output_path = '../output/'
model_path = '../model/'<jupyter_output><empty_output><jupyter_text>Inner-cellular mode choice will be dependent on cell-specific network data as well as trip-specific and socio-economic data. All categories must be available in the calibration data set as well as in the model.* Accessibility by mode in this cell (weighted average, see prep3X)* Population density (population by km²)* Road network coverage (length by km²) -- not implemented* Economic status of household or cell (GDP)* Car ownership in this cell* Employed persons in this cell  Load trips table from MiD and filter<jupyter_code>trips = pd.read_csv('../input/transport_demand/MiD2017_Regional_Wege.csv', sep=';')
trips = trips[['KKZ_SO', 'KKZ_ZO', 'wegkm_imp', 'km_routing_gr','wegmin_imp1',
               'hwzweck1', 'hvm_diff2', 'auto', 'W_SO2', 'sieddichte', # alternatively ewqkm
               'bahn28', 'bus28', 'bahn28_zo', 'bus28_zo']]
trips.columns = ['origin', 'destination', 'distance', 'distance_routed', 'duration',
                 'purpose', 'mode', 'car_avail', 'W_SO2', 'pop_density',
                 'dist_train_origin', 'dist_bus_origin', 'dist_train_destination', 'dist_bus_destination']
trips['distance'] = pd.to_numeric(trips['distance'].str.replace(',', '.'))
print(trips.shape)
# Filter for inner-German trips
trips = trips.loc[(trips['W_SO2']!=2) & (trips['W_SO2']!=3)]
print(trips.shape)
# Filter unspecified rows
trips = trips.loc[(trips['purpose']!=99) &
                  (trips['mode']<24)]
print(trips.shape)
# Trip distance
trips['distance'].plot.hist(bins=100, log=True, title='Trip distance distribution, logarithmic')
trips.loc[trips['distance']<=1].shape
# Filter for short-distance traffic
trips = trips.loc[~trips['distance_routed'].isin([8,9])]
trips = trips.loc[trips['distance']<=50]
print(trips.shape)
# Trip distance
trips['distance'].plot.hist(bins=100, log=True, title='Trip distance distribution, logarithmic')
# Compare to trip distance of inner-cell trips
# It should look roughly the same as above with lower total numbers though
trips.loc[(trips['origin']==trips['destination']) &
          (trips['origin']<9999) &
          (trips['destination']<9999), 'distance'].plot.hist(bins=100, log=True)
# Remove ferries
trips = trips.loc[trips['mode']!=19]
print(trips.shape)
# Match mode with the model modes
# Non-motorised modes are merged
# See cal10
model_modes = {1: 'rail_short', 2: 'rail_long',
               3: 'coach', 4: 'bus', 5: 'air', 6: 'car', 7: 'walk'}
mode_mid_vp_dict = {1:7, 2:7, 3:7, 4:6, 5:6, 6:6, 7:6, 8:6,
                    9:6, 10:6, 11:6, 12:4, 13:4, 14:4, 15:4,
                    16:1, 17:6, 18:6, 19:6, 20:2, 21:3, 22:3,
                    23:5, 24:7}
trips['mode_model'] = trips['mode'].map(mode_mid_vp_dict)
# Match trip purpose with VP2030 categories
# Note: Holiday trips are included in private (6)
# see cal10
purpose_mid_vp_dict = {1: 1, 2: 4, 3: 2, 4: 3, 5: 3, 6: 6, 7: 6, 99: 6}
trips['purpose_vp'] = trips['purpose'].map(purpose_mid_vp_dict)
# Convert distances to integer kilometers
dist_dict = {1:1, 2:1, 3:1, 4:2, 5:4, 6:6, 95:20, 701:20}
trips['dist_train_origin'] = trips['dist_train_origin'].map(dist_dict)
trips['dist_bus_origin'] = trips['dist_bus_origin'].map(dist_dict)
trips['dist_train_destination'] = trips['dist_train_destination'].map(dist_dict)
trips['dist_bus_destination'] = trips['dist_bus_destination'].map(dist_dict)
# Trip duration
trips['duration'].plot.hist(bins=100, log=True, title='Trip duration distribution, logarithmic')
# Trip purpose
trips.groupby('purpose_vp').size().plot.pie(
    subplots=False,figsize=(3, 3),labels=['work', 'education', 'shopping', 'official', 'private'],
    title='share of trips by purpose')
# Modal split
axs = trips.groupby(['mode_model','purpose_vp']).size().unstack(fill_value=0).rename(
    columns={1:'Commuting', 2:'Education', 3:'Shopping/medical', 4:'Business', 6:'Private'}).plot.pie(
    subplots=True, figsize=(16, 4),# labels=trips['mode_model'].map(model_modes),
    startangle=0, counterclock=False,
    legend=False, labeldistance=None)
axs[0].legend()
str(model_modes)
# Remove air, rail long and coach because they are long-distance only
trips = trips.loc[(trips['mode_model']!=2) &
                  (trips['mode_model']!=3) &
                  (trips['mode_model']!=5)]
print(trips.shape)<jupyter_output>(843582, 16)<jupyter_text>Imputate network data to trips<jupyter_code>sm = stepmodel.read_json(model_path + 'de_zones')
# Replace LAU IDs with NUTS IDs in origin and destination
lau_nuts_dict = sm.zones.set_index('lau_id')['NUTS_ID'].to_dict()
trips['NUTS_ID'] = trips['origin'].astype(str).map(lau_nuts_dict)
# Drop all other trips
trips = trips.loc[trips['NUTS_ID'].notna()]
print(trips.shape)
# Number of zones with survey participants
len(trips['NUTS_ID'].unique())<jupyter_output><empty_output><jupyter_text>Car ownershipCar ownership from the German federal statistical office GENESIS data base table 46251-0006: "Kraftfahrzeugbestand: Kreise, Stichtag, Kraftfahrzeugarten". Download at Nov. 26th 2020. License: Datenlizenz by-2-0; www.govdata.de/dl-de/by-2-0<jupyter_code># Load the table
cars = pd.read_excel(input_path + 'spatial_car_ownership_2017_GENESIS.xlsx')
cars['sum_MIT'] = cars['Krafträder'] + cars['Pkw']
cars['lau_id'] = cars['lau_id'].astype(str)
cars['sum_MIT'] = cars['sum_MIT'].replace({'--': 0}).astype(int)
cars.set_index('lau_id', drop=True, inplace=True)
lau_car_dict = cars['sum_MIT'].to_dict()
sm.zones['car_ownership'] = sm.zones['lau_id'].map(lau_car_dict)
sm.zones.head(2)
# Empty rows
print(len(sm.zones.loc[sm.zones['car_ownership'].isna()]))
sm.zones.loc[sm.zones['car_ownership']<sm.zones['population']/3]
# Car density
sm.zones['car_density'] = sm.zones['car_ownership'] / sm.zones['population']
# Replace empty rows with average density
sm.zones.loc[sm.zones['car_ownership']==0, 'car_density'] = \
    sm.zones['car_density'].replace({0: np.nan}).mean(skipna=True)
# Imputate into trips
zone_dict = sm.zones['car_density'].to_dict()
trips['car_density'] = trips['NUTS_ID'].map(zone_dict)<jupyter_output><empty_output><jupyter_text>Population density<jupyter_code># If not available for this trip, take average from model
print(trips.loc[trips['pop_density']==95].shape)
# Population density in the model
sm.zones['pop_density'] = sm.zones['population'] / sm.zones['area']
# Categorise like in MiD2017
bins = [0,100,250,500,1000,2000,5000,10000,100000]
cats = [1,2,3,4,5,6,7,8]
sm.zones['pop_density_cat'] = pd.cut(sm.zones['pop_density'], bins=bins, labels=cats)
zone_dict = sm.zones['pop_density_cat'].to_dict()
trips.loc[trips['pop_density']==95, 'pop_density'] = \
    trips.loc[trips['pop_density']==95, 'NUTS_ID'].map(zone_dict)
trips['pop_density'].hist(bins=50)
sm.zones['pop_density_cat'].hist(bins=50)<jupyter_output><empty_output><jupyter_text>AccessibilityEnrich trips with the accessibility metric<jupyter_code>def accessibility(dist_origin, dist_destination, weight=20):
    '''calculate an accessibility metric based on distances
    :param dist_origin: distance to the next mean of transport from origin in km
    :param dist_destination: distance from the used mean of transport to destination in km
    :param weight: (optional) inverse weight of distances
    :return: accessibility factor on a range from 0 to 1'''
    return max(0, 1 - dist_origin/weight - dist_destination/weight)
# The maximum accessibility for foot, bicycle and car is 1
trips['accessibility_walk'] = 1
trips['accessibility_car'] = trips['car_avail'].replace({9:0})
# Rail
trips['accessibility_rail'] = [accessibility(o, d) for o, d in zip(
    trips['dist_train_origin'], trips['dist_train_destination'])]
# Bus
trips['accessibility_bus'] = [accessibility(o, d) for o, d in zip(
    trips['dist_bus_origin'], trips['dist_bus_destination'])]
# Summary
trips.groupby('mode_model').agg(
    {'accessibility_car': 'mean', 'accessibility_rail': 'mean',
     'accessibility_bus': 'mean'})<jupyter_output><empty_output><jupyter_text>GDPTake cell-specific GDP data from eurostat https://ec.europa.eu/eurostat/de/web/rural-development/data<jupyter_code>pd.read_csv(input_path + 'spatial_employed_persons_2017_eurostat.csv', encoding='latin-1')
gdp = pd.read_csv(input_path + 'spatial_GDP_2017_eurostat.csv', encoding='latin-1')
gdp['Value'] = gdp['Value'].str.replace(',', '').astype(float)
# Assign GDP to zones
gdp_dict = gdp.loc[gdp['UNIT']=='Euro per inhabitant'].set_index('GEO')['Value'].to_dict()
sm.zones['gdp_pc'] = sm.zones['NUTS_NAME'].map(gdp_dict)
sm.zones.loc[sm.zones['gdp_pc'].isna(), 'gdp_pc'] = \
    sm.zones.loc[sm.zones['gdp_pc'].notna(), 'gdp_pc'].mean()
sm.zones['gdp_pc'] = sm.zones['gdp_pc'].astype(int)
sm.zones.loc[sm.zones['gdp_pc'].isna()]<jupyter_output><empty_output><jupyter_text>Save dataset<jupyter_code>trips.to_csv('../input/transport_demand/calibration_intra-cellular_trips_MiD2017.csv')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16313284292514063, 'fraction_numerical': 0.032994374729554306, 'mean_word_length': 6.111538461538461, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 32, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '615052', 'license': 'Apache-2.0', 'n_tokens_mistral': 3559, 'n_tokens_neox': 3275, 'n_words': 681, 'path': 'quetzal_germany-master/notebooks/cal13_inner-zone_data.ipynb', 'repo_name': 'avi-shake/Flensburg_Modeling', 'size': '9244'}