<jupyter_start><jupyter_text>First let's collect some traffic data for Hamburg.The meta data can be found at the [Transparenzportal Hamburg](http://suche.transparenz.hamburg.de/dataset/verkehrslage-auf-autobahnen-schleifen-hamburg11?forceWeb=true) or at [metaver](http://metaver.de/trefferanzeige?docuuid=2FC4BBED-350C-4380-B138-4222C28F56C6).Visualisation are offered at both places, i.e. [Transparenzportal Hamburg](http://www.geoportal-hamburg.de/Geoportal/geo-online/?mdid=2FC4BBED-350C-4380-B138-4222C28F56C6) and [metaver](http://metaver.de/kartendienste?lang=en&topic=anbieter&bgLayer=webatlasde_light&E=552822.61&N=5926524.84&zoom=6&layers=WMS%7C%7CDWiStA%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Cdwista%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CHamburger%2520Autobahnabschnitte%2520ohne%2520Verkehrslagedaten%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Cbab_novkl%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CVerkehrslage%2520auf%2520Autobahnen%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Cbab_vkl%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CParkh%25C3%25A4user%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Cparkhaeuser%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CReisebusparkpl%25C3%25A4tze%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Creisebusparkpl%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CBike%2520%252B%2520Ride%2520Anlagen%2520Hamburg%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Cbikeandride%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CPark%2520%252B%2520Ride%2520Anlagen%2520Hamburg%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Cparkandride%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CBehindertenparkpl%25C3%25A4tze%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Cbehindertenparkpl%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CMotorradparkpl%25C3%25A4tze%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Cmotorradparkpl%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CVerkehrskameras%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Cverkehrskameras%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CBaustellen%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Cbaustellen_prod%7C%7C1.3.0%7C%7Ctrue%2CWMS%7C%7CFachdaten%7C%7Chttps%3A%2F%2Fgeodienste.hamburg.de%2FHH_WMS_Verkehr_opendata%3F%7C%7Ctheme_hh_verkehrsdaten%7C%7C1.3.0%7C%7C&layers_visibility=false,false,true,false,false,false,false,false,false,false,false,false).<jupyter_code>url = "http://transport_verkehr.daten-hamburg.de/vkl/vkl.txt"
resp = requests.get(url)
content = resp.text
content<jupyter_output><empty_output><jupyter_text>So let's parse that data<jupyter_code>lines = [line.split(";") for line in content.split("#\r\n") if line]
metadata = lines[:6]
data = lines[6:]

print("Metadata: ", metadata)
print("Data: ", data)
translation = {
    "0": "flowing",
    "1": "stop-and-go",
    "2": "heavy",
    "3": "congestion",
    "255": "no data"
}

df = pd.DataFrame(data={
    'location_id': [int(row[0]) for row in data],
    'state_of_traffic': [translation[row[1]] for row in data]
})

df = df.set_index('location_id')
df = df.sort_index()
df<jupyter_output><empty_output><jupyter_text>Next task is to gather the geopositions of the locations and visualise them.<jupyter_code>resp = requests.get("https://geodienste.hamburg.de/HH_WFS_Verkehr_opendata?"
                    + "SERVICE=WFS&REQUEST=GetFeature&VERSION=1.1.0&TYPENAME=bab_vkl")
content = resp.text
print(content)
import xml.etree.ElementTree as ET
tree = ET.fromstring(content)
tree
data = {
    'location_id': [],
    'coordinates': [],
    'state_of_traffic_2': []
}

namespaces = {"app": "http://www.deegree.org/app", "gml": "http://www.opengis.net/gml"}

for node in tree.findall("./gml:featureMember/app:bab_vkl", namespaces=namespaces):
    vkl_id = int(node.find("app:vkl_id", namespaces=namespaces).text)
    klasse_node = node.find("app:klasse", namespaces=namespaces)
    klasse = klasse_node.text if klasse_node is not None else ""
    klasse_translated = translation.get(klasse, "no data")
    location_node = node.find("./app:the_geom/gml:MultiLineString/gml:lineStringMember/gml:LineString/gml:posList", 
                         namespaces=namespaces)
    location = location_node.text if location_node is not None else ""
    data["location_id"].append(vkl_id)
    data["coordinates"].append(location)
    data["state_of_traffic_2"].append(klasse_translated)
    
df2 = pd.DataFrame(data=data)
df2 = df2.set_index("location_id")
df2 = df2.sort_index()
df2
big_df = df.join(df2)
big_df<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15035339473120582, 'fraction_numerical': 0.0843863782394517, 'mean_word_length': 8.47261663286004, 'pattern_counts': {'":': 7, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 1, 'lorem ipsum': 0, 'www.': 3, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '620496', 'license': 'MIT', 'n_tokens_mistral': 2313, 'n_tokens_neox': 2244, 'n_words': 166, 'path': 'Use open traffic data of Hamburg.ipynb', 'repo_name': '1kastner/verkehrsdaten', 'size': '4669'}