<jupyter_start><jupyter_text>Step 2: Building GTFS graphs and merging it with a walking graphWe heavily follow <NAME>'s Calculating Betweenness Centrality with GTFS blog post: https://gist.github.com/kuanb/c54d0ae7ee353cac3d56371d3491cf56 The peartree (https://github.com/kuanb/peartree) source code was modified. Until code is merged you should use code from this fork: https://github.com/d3netxer/peartree<jupyter_code>%load_ext autoreload
%autoreload 2
import matplotlib.pyplot as plt
%matplotlib inline
import osmnx as ox
import pandas as pd
import geopandas as gpd
import networkx as nx
import numpy as np
from shapely.geometry import Point
import partridge as ptg
import os, sys
sys.path.append(r"C:\repos\peartree")
import peartree as pt
print(pt.__file__)
path = r'input_folder/cap_haitien_gtfs.zip'<jupyter_output><empty_output><jupyter_text>Build a graph from service_0001service_0001 is on the weekends, so below we are choosing a data that lands on a weekend<jupyter_code># from: http://simplistic.me/playing-with-gtfs.html
import datetime

service_ids_by_date = ptg.read_service_ids_by_date(path)
service_ids = service_ids_by_date[datetime.date(2019, 6, 29)]

print(f"service_ids is {service_ids}")

# view lets you filter before you load the feed. For example, below you are filtering by the service_ids
feed_0001 = ptg.load_feed(path, view={
    'trips.txt': {
        'service_id': service_ids,
    },
})
feed_0001.calendar<jupyter_output><empty_output><jupyter_text>give all trips a direction of 0PearTree wants directions assigned<jupyter_code>feed_0001.trips['direction_id'] = 0<jupyter_output><empty_output><jupyter_text>Preview the GTFS network<jupyter_code># Set a target time period to summarize impedance
start = 0  # 0:00 
end = 24*60*60  # 24:00 

# Converts feed subset into a directed
# network multigraph
G = pt.load_feed_as_graph(feed_0001, start, end, add_trips_per_edge=True)
fig, ax = ox.plot_graph(G,
                            figsize=(12,12),
                            show=False,
                            close=False,
                            node_color='#8aedfc',
                            node_size=5,
                            edge_color='#e2dede',
                            edge_alpha=0.25,
                            bgcolor='black')
# PearTree prepends the stop ids with a code the is different each time it loads a graph
list(G.edges)
#list(G.edges(data='True'))
len(G.nodes)<jupyter_output><empty_output><jupyter_text>Inspect edge data, and you should see the length attribute, which is the time in seconds needs to traverse an edge. The trips attribute represents how many trips cross that edge.<jupyter_code>for edge in list(G.edges):
    print(G.get_edge_data(edge[0],edge[1]))<jupyter_output>{0: {'length': 2119.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 1462.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 74.0, 'trips': 2, 'mode': 'transit'}}
{0: {'length': 122.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 706.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 1213.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 648.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 1257.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 2657.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 1736.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 90.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 40.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 64.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 26.5, 'trips': 2, 'mode': 'transit'}}
{0: {'length': 94.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 66.25, 'trips': 4, 'mode': 'transit'}}
{0: {'length': 345.0, 'trips': 1, 'mode': 'transit'}}
{0: {'length': 108.5, 'trips': 4, 'mode': 'transit'}}
{0: {'length': 137.0, 'trips[...]<jupyter_text>get feed 2<jupyter_code>service_ids_by_date = ptg.read_service_ids_by_date(path)
service_ids = service_ids_by_date[datetime.date(2019,8,6)]

print(f"service_ids is {service_ids}")

# view lets you filter before you load the feed. For example, below you are filtering by the service_ids
feed_0002 = ptg.load_feed(path, view={
    'trips.txt': {
        'service_id': service_ids,
    },
})<jupyter_output>service_ids is frozenset({'service_0002'})<jupyter_text>Inspect graph as a shapefileUsed for testing<jupyter_code># Get reference to GOSTNets
#sys.path.append(r'C:\repos\GOSTnets')
#import GOSTnets as gn
#gn.save(G,"gtfs_export_cap_haitien_service0001",r"temp")
#gn.save(G,"gtfs_export_cap_haitien_service0002",r"temp")
# Also these saved edges will be used in the optional PostProcessing notebook to compare differences between the two graphs<jupyter_output><empty_output><jupyter_text>note: On inspection the edges have a length field. This length field is the average traversal time per edge based on the GTFS data in seconds.  Merge a walk networkfollowing this blog post: http://kuanbutts.com/2018/12/24/peartree-with-walk-network/<jupyter_code># load existing walk/ferry graph from step 1

G = nx.read_gpickle(r"temp\cap_haitien_walk_w_ferries_via_osmnx_origins_adv_snap.pickle")
#G = nx.read_gpickle(r"temp\cap_haitien_walk_w_ferries_via_osmnx_salted.pickle")
print(nx.info(G))
list(G.edges(data=True))[:10]<jupyter_output><empty_output><jupyter_text>Assign traversal times in seconds to edgesSince peartree represents edge length (that is the impedance value associated with the edge) in seconds; we will need to convert the edge values that are in meters into seconds:<jupyter_code>walk_speed = 3.5 #km per hour; about 3 miles per hour
ferry_speed = 15

# Make a copy of the graph in case we make a mistake
G_adj = G.copy()

# Iterate through and convert lengths to seconds
for from_node, to_node, edge in G_adj.edges(data=True):
    orig_len = edge['length']
       
    # Note that this is a MultiDiGraph so there could
    # be multiple indices here, I naively assume this is not the case
    G_adj[from_node][to_node][0]['orig_length'] = orig_len
     
    try:
        # if ferry
        if 'ferry' in G_adj[from_node][to_node][0]:
            ferry_var = G_adj[from_node][to_node][0]['ferry']
            # if ferry does not have nan as a value
            # if it is a string then it will produce an error and go to the except statement
#             print('print ferry_var')
#             print(ferry_var)
#             print(type(ferry_var))
#             print(np.isnan(ferry_var))
            if not np.isnan(ferry_var):
                print(G_adj[from_node][to_node][0]['ferry'])
                print(G_adj[from_node][to_node][0])

                # Conversion of walk speed and into seconds from meters
                kmph = (orig_len / 1000) / ferry_speed
                in_seconds = kmph * 60 * 60
                G_adj[from_node][to_node][0]['length'] = in_seconds

                # And state the mode, too
                G_adj[from_node][to_node][0]['mode'] = 'ferry'
            else:
                # Conversion of walk speed and into seconds from meters
                kmph = (orig_len / 1000) / walk_speed
                in_seconds = kmph * 60 * 60
                G_adj[from_node][to_node][0]['length'] = in_seconds

                # And state the mode, too
                G_adj[from_node][to_node][0]['mode'] = 'walk'
    except:
        # Conversion of walk speed and into seconds from meters
        kmph = (orig_len / 1000) / walk_speed
        in_seconds = kmph * 60 * 60
        G_adj[from_node][to_node][0]['length'] = in_seconds
    
        # And state the mode, too
        G_adj[from_node][to_node][0]['mode'] = 'walk'
G_adj.nodes[330530920]
G_adj.nodes[6770195160]
# So this should be easy - just go through all nodes
# and make them have a 0 cost to board
for i, node in G_adj.nodes(data=True):
    G_adj.nodes[i]['boarding_cost'] = 0
# testing
list(G_adj.edges(data=True))[1]<jupyter_output><empty_output><jupyter_text>save the graph again to be used for the isochrones notebook<jupyter_code>sys.path.append(r'C:\repos\GOSTnets')
import GOSTnets as gn
gn.save(G,"cap_haitien_walk_w_ferries_via_osmnx_w_time_adv_snap",r"temp")<jupyter_output><empty_output><jupyter_text>Loading the feeds as graphs with the walking graph as the existing graphNow that the two graphs have the same internal structures, we can load the walk network onto the transit network with the following peartree helper method.<jupyter_code># Now that we have a formatted walk network
# it should be easy to reload the peartree graph
# and stack it on the walk network
start = 0  # 0:00 
end = 24*60*60  # 24:00 

feeds = {'service0001':feed_0001,'service0002':feed_0002}
#feeds = {'service0002':feed_0002}
for feed in feeds.items():
    G_adj_copy = G_adj.copy()
    # Note this will be a little slow - an optimization here would be
    # to have coalesced the walk network
    %time G = pt.load_feed_as_graph(feed[1], start, end, existing_graph=G_adj_copy, impute_walk_transfers=True, add_trips_per_edge=True)
    
    # compatible with NetworkX 2.4
    list_of_subgraphs = list(G.subgraph(c).copy() for c in nx.weakly_connected_components(G))
    max_graph = None
    max_edges = 0
    for i in list_of_subgraphs:
        if i.number_of_edges() > max_edges:
            max_edges = i.number_of_edges()
            max_graph = i

    # set your graph equal to the largest sub-graph
    G = max_graph
    
    # save again and inspect
    gn.save(G,f"gtfs_export_cap_haitien_merged_impute_walk_adv_snap_{feed[0]}",r"temp")
    #gn.save(G,f"gtfs_export_cap_haitien_merged_impute_walk_salted_{feed[0]}",r"temp")<jupyter_output>Wall time: 32.7 s
Wall time: 30.8 s<jupyter_text>Visualize the last merged feed in the loop<jupyter_code>G.graph['crs'] = 'epsg:4326'
G.graph
G.nodes[6770195160]
fig, ax = ox.plot_graph(G,
                            figsize=(12,12),
                            show=False,
                            close=False,
                            node_color='#8aedfc',
                            node_size=5,
                            edge_color='#e2dede',
                            edge_alpha=0.25,
                            bgcolor='black')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11874134177716208, 'fraction_numerical': 0.03601820700573916, 'mean_word_length': 3.514068780705672, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 48, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '11270', 'license': 'MIT', 'n_tokens_mistral': 3725, 'n_tokens_neox': 3431, 'n_words': 939, 'path': 'Implementations/FY21/ACC_GTFS_accessibility_analysis_Cap_Haitien_Haiti/Step2_build_merge_gtfs_graphs_adv_snap.ipynb', 'repo_name': 'worldbank/GOST_PublicGoods', 'size': '10110'}