<jupyter_start><jupyter_text>Cleaning OpenRoads topologyOpenRoads data are not always topologically correct, meaning that they introduce nodes of a degree 2. Topologically precise network has a single LineString between two junctions. This notebook fixes topology of OpenRoads data and saves resulting GeoDataFrame to PostGIS.Note that `consolidate` is a [custom python file](https://github.com/urbangrammarai/data_processing/blob/master/vector_data/consolidate.py) containing definitions of required functions. Even though these were originally written for this project, in future they will be included in the existing Python ecosystem:- `topology` will be included in [momepy 0.4](https://github.com/martinfleis/momepy) as `remove_false_nodes`<jupyter_code>import os
import time

import geopandas as gpd

from sqlalchemy import create_engine

from consolidate import topology

user = os.environ.get('DB_USER')
pwd = <PASSWORD>('<PASSWORD>')
host = os.environ.get('DB_HOST')
port = os.environ.get('DB_PORT')
db_connection_url = f"postgres+psycopg2://{user}:{pwd}@{host}:{port}/built_env"
engine = create_engine(db_connection_url)
engine.begin()<jupyter_output><empty_output><jupyter_text>We load the whole `openroads_200803` table from existing PostGIS database.<jupyter_code>df = gpd.read_postgis('SELECT * FROM openroads_200803', engine, geom_col='geometry')
df.shape<jupyter_output><empty_output><jupyter_text>Using `topology` function stored in `consolidation.py` file, we clean the network by merging segments which introduces false junctions.<jupyter_code>s = time.time()
fixed = topology(df)
print(time.time() - s)
fixed.shape
fixed.to_postgis("openroads_200803_topological", engine, if_exists='replace')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09644087256027555, 'fraction_numerical': 0.012629161882893225, 'mean_word_length': 7.420289855072464, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '413315', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 561, 'n_tokens_neox': 529, 'n_words': 151, 'path': 'vector_data/OpenRoads_Topology.ipynb', 'repo_name': 'darribas/data_processing', 'size': '1742'}