<jupyter_start><jupyter_text>Example of data analysis based on vehicles registrations in LuxembourgThe data is coming from SNCA and published as open data on data.public.lu:https://data.public.lu/fr/datasets/parc-automobile-du-luxembourg/Let's import the data from the last months.<jupyter_code>import requests
import re
import xml.etree.ElementTree as ET
import pandas as pd
import matplotlib.pyplot as plt
import time

### install
# Install libraries with the following command: pip install -r requirements.txt

### config

# your API key here
key = ''

# the id of the Dataset you want to get
datasetId = 'parc-automobile-du-luxembourg'

# the pattern filenames should match to be downloaded

#fPattern = '^Operations_Delta_\\d{6}.xml$'
#fPattern = '^Parc_Automobile_20220\\d.xml$'
fPattern = '^Parc_Automobile_202204.xml$'
### end config

url = 'https://data.public.lu/api/1/datasets/'+datasetId+'/'
headers = {'X-API-KEY': key}
r = requests.get(url, headers=headers)
r.raise_for_status()

resources = r.json()['resources']
data = []
columns = []
init = True
for i in range(len(resources)):
    if re.search(fPattern, resources[i]['title']):
        print('Downloading: '+ resources[i]['title'])
        dlStart = time.time()
        s = requests.get(resources[i]['url'], headers=headers)
        s.raise_for_status()
        dlEnd = time.time()
        print('Downloaded! download duration:', dlEnd-dlStart)
        print('Parsing...')
        xml = ET.fromstring(s.text)
        parseEnd = time.time()
        print('Parsed! parsing duration:', parseEnd-dlEnd)
        for operation in xml:
            row = []
            for field in operation:
                if init:
                    columns.append(field.tag)
                row.append(field.text)
            init = False
            data.append(row)
df = pd.DataFrame(data, columns=columns)<jupyter_output>Downloading: Parc_Automobile_202204.xml
Downloaded! download duration: 30.619593143463135
Parsing...
Parsed! parsing duration: 2923.9113500118256<jupyter_text>Ok now we have some data, let's filter to get only private cars<jupyter_code>cars = df[df["CATSTC"] == "5"]<jupyter_output><empty_output><jupyter_text>Let's find the most frequent colors of cars in Luxembourg on the given period of time.<jupyter_code>coul = cars['COUL'].value_counts(normalize=True).head(10)
coul.plot.pie(figsize=(6, 6))<jupyter_output><empty_output><jupyter_text>Most frequent engine types<jupyter_code>energy = cars['LIBCRB'].value_counts(normalize=True)
energy.plot.bar()<jupyter_output><empty_output><jupyter_text>Most frequent brands<jupyter_code>brand = cars['LIBMRQ'].value_counts(normalize=True).head(30)
brand.plot.bar()<jupyter_output><empty_output><jupyter_text>Vehicles Frequency by CO2 emissions<jupyter_code>co2 = cars['INFCO2'].dropna().astype('int').plot.hist(bins=20)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12844677137870855, 'fraction_numerical': 0.02268760907504363, 'mean_word_length': 4.387218045112782, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 7}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '219237', 'license': 'MIT', 'n_tokens_mistral': 993, 'n_tokens_neox': 940, 'n_words': 210, 'path': 'parc_automobile.ipynb', 'repo_name': 'AlainVagner/odAnalysisExampleSNCA', 'size': '2865'}