<jupyter_start><jupyter_text>Descubrimientos de patrones de accidentes de tránsito en la CDMX  Generación de características espaciales indirectas Cargamos las librerías necesarias<jupyter_code>from math import radians, cos, sin, asin, sqrt, atan2
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib as mpl
from pylab import rcParams
import seaborn as sns
import folium
import networkx as nx
import osmnx as ox
import geopandas as gpd
from shapely.geometry import Point, Polygon, LineString, MultiLineString
import shapefile as shp
import warnings
warnings.filterwarnings('ignore')
pd.set_option('display.max_columns', None)
sns.mpl.rc("figure", figsize=(20, 15))
mpl.rcParams['figure.dpi']= 200<jupyter_output><empty_output><jupyter_text>Cargamos las capas a usar para generar las características espaciales previamente procesadas<jupyter_code>pois         = gpd.read_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//shp_b//processed_data//shp_OSM//pois.shp')
traffic      = gpd.read_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//shp_b//processed_data//shp_OSM//traffic.shp')
trans        = gpd.read_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//shp_b//processed_data//shp_transp//estaciones_transp.shp')
intersect    = gpd.read_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//shp_b//processed_data//shp_OSM//intersections.shp')
road_hex50   = gpd.read_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//shp_b//processed_data//hex_roads//road_hex50.shp')
road_hex100  = gpd.read_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//shp_b//processed_data//hex_roads//road_hex100.shp')
road_hex200  = gpd.read_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//shp_b//processed_data//hex_roads//road_hex200.shp')
road_hex300  = gpd.read_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//shp_b//processed_data//hex_roads//road_hex300.shp')
road_hex500  = gpd.read_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//shp_b//processed_data//hex_roads//road_hex500.shp')
cruces_index = gpd.read_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//shp_b//processed_data//cruces//cruces_pelis.shp')
road_hex50 = road_hex50.reset_index()
road_hex50.rename(columns={'index':'FID'}, inplace = True)

road_hex100 = road_hex100.reset_index()
road_hex100.rename(columns={'index':'FID'}, inplace = True)

road_hex200 = road_hex200.reset_index()
road_hex200.rename(columns={'index':'FID'}, inplace = True)

road_hex300 = road_hex300.reset_index()
road_hex300.rename(columns={'index':'FID'}, inplace = True)

road_hex500 = road_hex500.reset_index()
road_hex500.rename(columns={'index':'FID'}, inplace = True)<jupyter_output><empty_output><jupyter_text>Vemos cuantos hexagonos tiene cada malla con distintos radios<jupyter_code>print("La malla con radio de 50m contiene:",len(road_hex50), "hexágonos")
print("La malla con radio de 100m contiene:",len(road_hex100), "hexágonos")
print("La malla con radio de 200m contiene:",len(road_hex200), "hexágonos")
print("La malla con radio de 300m contiene:",len(road_hex300), "hexágonos")
print("La malla con radio de 500m contiene:",len(road_hex500), "hexágonos")<jupyter_output>La malla con radio de 50m contiene: 263405 héxagonos
La malla con radio de 50m contiene: 94510 héxagonos
La malla con radio de 50m contiene: 25859 héxagonos
La malla con radio de 50m contiene: 13043 héxagonos
La malla con radio de 50m contiene: 5501 héxagonos<jupyter_text>Para cada malla hexagonal de distinto radio, debemos de agregar las características indirectas como sigue<jupyter_code>g = road_hex200<jupyter_output><empty_output><jupyter_text>Agregamos las caracteristicas indirectas al grid Función para la agregación de las variables indirectas a la malla, donde:    1. g --> es la malla a la que se agregaran las características indirectas.    2. p --> es la característica indirecta que se agregará.    3. fclass --> es el tipo de característica indirecta.<jupyter_code>def agreg_ind(g, p, fclass):
    p_x = p[p.fclass == fclass]
    x = p_x[['geometry']].copy()
    g = gpd.sjoin(g, x, how = 'left', op = 'intersects')
    g = g.fillna(0)
    g = g.rename(columns = {'index_right':fclass})
    g[fclass] = g[fclass].apply(lambda x: 0 if x == 0.0 else 1)
    g = g.groupby('FID', as_index = False).agg({fclass: 'sum', 
                                           'geometry':'first'})
    g = gpd.GeoDataFrame(g, crs = '4326')
    print("Característica indirecta agregada",fclass,g.shape)
    return g<jupyter_output><empty_output><jupyter_text>Agregamos los puntos de interés a malla hexagonal<jupyter_code>lst_pois = ['university',
            'school',
            'kindergarten',
            'college',
            'theatre',
            'nightclub',
            'cinema',
            'park',
            'restaurant',
            'pub',
            'bar',
            'supermarket',
            'mall',
            'bank',
            'museum',
            'attraction',
            'hospital',
            'hotel'
           ]


p = pois

lst_p = []
for i in lst_pois:
    e = agreg_ind(g, p, i)
    lst_p.append(e)
    
universidad        = lst_p[0]    
escuela            = lst_p[1]
kinder             = lst_p[2]
secundaria         = lst_p[3]
teatro             = lst_p[4]
club_nocturno      = lst_p[5]
cine               = lst_p[6]
parque             = lst_p[7]
restaurante        = lst_p[8]
pub                = lst_p[9]
bar                = lst_p[10]
supermercado       = lst_p[11]
centro_comercial   = lst_p[12]
banco              = lst_p[13]
museo              = lst_p[14]
atraccion_turstica = lst_p[15]
hospital           = lst_p[16]              
hotel              = lst_p[17]<jupyter_output>Característica indirecta agregada university (25859, 3)
Característica indirecta agregada school (25859, 3)
Característica indirecta agregada kindergarten (25859, 3)
Característica indirecta agregada college (25859, 3)
Característica indirecta agregada theatre (25859, 3)
Característica indirecta agregada nightclub (25859, 3)
Característica indirecta agregada cinema (25859, 3)
Característica indirecta agregada park (25859, 3)
Característica indirecta agregada restaurant (25859, 3)
Característica indirecta agregada pub (25859, 3)
Característica indirecta agregada bar (25859, 3)
Característica indirecta agregada supermarket (25859, 3)
Característica indirecta agregada mall (25859, 3)
Característica indirecta agregada bank (25859, 3)
Característica indirecta agregada museum (25859, 3)
Característica indirecta agregada attraction (25859, 3)
Característica indirecta agregada hospital (25859, 3)
Característica indirecta agregada hotel (25859, 3)<jupyter_text>Agregamos infraestructura vial a malla hexagonal<jupyter_code>lst_traffic = ['traffic_signals',
               'mini_roundabout',
               'crossing',
               'turning_circle',
               'speed_camera',
               ]


p = traffic

lst_t = []
for i in lst_traffic:
    e = agreg_ind(g, p, i)
    lst_t.append(e)
    
semaforos         = lst_t[0]    
glorietas         = lst_t[1]
cruces_peatonales = lst_t[2]
retornos          = lst_t[3]
camaras_velocidad = lst_t[4]<jupyter_output>Característica indirecta agregada traffic_signals (25859, 3)
Característica indirecta agregada mini_roundabout (25859, 3)
Característica indirecta agregada crossing (25859, 3)
Característica indirecta agregada turning_circle (25859, 3)
Característica indirecta agregada speed_camera (25859, 3)<jupyter_text>Agregamos infraestructura de transporte (metro, metrobus, rtp, trolebus, CETRAMS) a malla hexagonal<jupyter_code>lst_trans = ['metro',
             'metrobus',
             'rtp',
             'trolebus',
             'cetram'
            ]


p = trans

lst_tr = []
for i in lst_trans:
    e = agreg_ind(g, p, i)
    lst_tr.append(e)
    
est_metro    = lst_tr[0]    
est_metrobus = lst_tr[1]
par_rtp      = lst_tr[2]
est_trolebus = lst_tr[3]
cetrams      = lst_tr[4]<jupyter_output>Característica indirecta agregada metro (25859, 3)
Característica indirecta agregada metrobus (25859, 3)
Característica indirecta agregada rtp (25859, 3)
Característica indirecta agregada trolebus (25859, 3)
Característica indirecta agregada cetram (25859, 3)<jupyter_text>Agregamos los cruces peligrosos a la malla hexagonal<jupyter_code>p = cruces_index
cruces_peligrosos = agreg_ind(g, p, 'cruce_peligroso')<jupyter_output>Característica indirecta agregada cruce_peligroso (25859, 3)<jupyter_text>Agregamos intersecciones a malla hexagonal<jupyter_code>p = intersect
interseccion = agreg_ind(g, p, 'interseccion')<jupyter_output>Característica indirecta agregada interseccion (25859, 3)<jupyter_text>Agregamos ciertas caracteristicas formando un buffer alrededor de una distancia dada Función para la agregación de las variables indirectas a la malla, con buffer donde:    1. g --> es la malla a la que se agregaran las características indirectas.    2. p --> es la característica indirecta que se agregará.    3. fclass --> es el tipo de característica indirecta.    4. m --> son los metros de buffer a partir de cada punto.<jupyter_code>def agreg_ind_dist(g,p,fclass,m,):
    p_x = p[p.fclass == fclass]
    x = p_x[['geometry']].copy()
    dist = (m * 0.1) / 11000
    x['geometry'] = x.geometry.buffer(dist)
    g = gpd.sjoin(g, x, how = 'left', op = 'intersects')
    g = g.fillna(0)
    fclass = fclass + '_' + str(m)
    g = g.rename(columns = {'index_right':fclass})
    g[fclass] = g[fclass].apply(lambda x: 0 if x == 0.0 else 1)
    g = g.groupby('FID', as_index = False).agg({fclass: 'sum', 
                                               'geometry':'first'})
    g = gpd.GeoDataFrame(g, crs = '4326')
    print("Característica indirecta agregada",fclass,g.shape)
    return g
lst_pois = ['school',
            'nightclub',
            'restaurant',
            'bar',
            'hospital',
            'hotel']


p = pois

lst_p = []
for i in lst_pois:
    e = agreg_ind_dist(g, p, i, 300)
    lst_p.append(e)

    
escuela300            = lst_p[0]  
club_nocturno300      = lst_p[1]
restaurante300        = lst_p[2]
bar300                = lst_p[3]
hospital300           = lst_p[4]              
hotel300              = lst_p[5]
lst_traffic = ['traffic_signals',
               'crossing',
               'speed_camera',
               ]


p = traffic

lst_t = []
for i in lst_traffic:
    e = agreg_ind_dist(g, p, i, 300)
    lst_t.append(e)
    
semaforos300         = lst_t[0]    
cruces_peatonales300 = lst_t[1]
camaras_velocidad300 = lst_t[2]

lst_trans = ['rtp',
             'metro']


p = trans

lst_tr = []
for i in lst_trans:
    e = agreg_ind_dist(g, p, i, 300)
    lst_tr.append(e)
    
est_metro300    = lst_tr[0]    
par_rtp300      = lst_tr[1]

def concat_dataset(df1, df2):
    data = pd.concat([df1, df2], axis = 1)
    _, i = np.unique(data.columns, return_index=True)
    data = data.iloc[:, i]
    return data
g.head()
len(g)
data = g.copy()
data["X"] = data.centroid.x
data["Y"] = data.centroid.y

data = concat_dataset(data, universidad)       
data = concat_dataset(data, escuela)           
data = concat_dataset(data, kinder)            
data = concat_dataset(data, secundaria)        
data = concat_dataset(data, teatro)            
data = concat_dataset(data, club_nocturno)     
data = concat_dataset(data, cine)              
data = concat_dataset(data, parque)            
data = concat_dataset(data, restaurante)       
data = concat_dataset(data, pub)               
data = concat_dataset(data, bar)               
data = concat_dataset(data, supermercado)      
data = concat_dataset(data, centro_comercial)  
data = concat_dataset(data, banco)             
data = concat_dataset(data, museo)             
data = concat_dataset(data, atraccion_turstica)
data = concat_dataset(data, hospital)          
data = concat_dataset(data, hotel) 
data = concat_dataset(data, escuela300)      
data = concat_dataset(data, club_nocturno300)
data = concat_dataset(data, restaurante300)  
data = concat_dataset(data, bar300)          
data = concat_dataset(data, hospital300)     
data = concat_dataset(data, hotel300)   
 
data = concat_dataset(data, semaforos)        
data = concat_dataset(data, glorietas)        
data = concat_dataset(data, cruces_peatonales)
data = concat_dataset(data, retornos)         
data = concat_dataset(data, camaras_velocidad)
data = concat_dataset(data, semaforos300)        
data = concat_dataset(data, cruces_peatonales300)
data = concat_dataset(data, camaras_velocidad300)

data = concat_dataset(data, est_metro)   
data = concat_dataset(data, est_metrobus)
data = concat_dataset(data, par_rtp)     
data = concat_dataset(data, est_trolebus)
data = concat_dataset(data, cetrams)  
data = concat_dataset(data, est_metro300)
data = concat_dataset(data, par_rtp300) 

data = concat_dataset(data, interseccion)
data = concat_dataset(data, cruces_peligrosos)
data = data[['X','Y','attraction', 'bank', 'bar', 'bar_300','cetram', 'cinema', 'college', 'crossing', 
             'crossing_300','cruce_peligroso','hospital', 'hospital_300', 'hotel','hotel_300', 'interseccion', 
             'kindergarten','mall', 'metro','metro_300', 'metrobus', 'mini_roundabout', 'museum', 'nightclub',
             'nightclub_300', 'park', 'pub', 'restaurant', 'restaurant_300','rtp', 'rtp_300', 'school', 
             'school_300', 'speed_camera','speed_camera_300', 'supermarket', 'theatre','traffic_signals',
             'traffic_signals_300', 'trolebus', 'turning_circle', 'university','geometry']]
data.info()
data.crs = {'init' :'epsg:4326'}
data.to_file('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//results_static//shp_ind_200m.shp')
dt = data.copy()
del dt['geometry']
dt.to_csv('/Users/daniel.rodriguez/Documents/ACC/ACC_PROOF/ACC1//results_static//csv_ind_200m.csv')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11431862356921747, 'fraction_numerical': 0.038730113022820534, 'mean_word_length': 3.6091572660915725, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 0, '>': 46, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '474867', 'license': 'MIT', 'n_tokens_mistral': 5510, 'n_tokens_neox': 5113, 'n_words': 1059, 'path': 'CODE/py3/Create indirect features.ipynb', 'repo_name': 'Daro05/tesis_accidentes', 'size': '13891'}