<jupyter_start><jupyter_text>Notebook objective This notebook's objective is to provide an automated way to download the dataset provided by the Whereismytransport team into a 'raw' data folder.This makes the analysis easier to replicate by team members and more robust as the manual manipulation of the dataset (eg download, save file) would no longer be required. Assessment source: https://drive.google.com/drive/folders/1akT7jbXYy2-zLHEtnK3q6vqcQ-JdRO9H  Preparation  Import required libraries<jupyter_code>import os
import gdown<jupyter_output><empty_output><jupyter_text>Folder structure<jupyter_code>input_folder = './data/raw/'
output_folder = './reports/figures/'<jupyter_output><empty_output><jupyter_text>Download datasets<jupyter_code>url_alerts = "https://drive.google.com/file/d/1u98UkVheVIcyXx-ir1Gp5fggsu2V3ztC/view?usp=sharing"
url_notifications = "https://drive.google.com/file/d/1KpW7s4_JfJ-rz2vT14AjL1WwPmI-3Zgy/view?usp=sharing"
os.chdir(input_folder)
gdown.download(url_alerts, fuzzy=True)
gdown.download(url_notifications, fuzzy=True)
os.chdir('..')<jupyter_output>Downloading...
From: https://drive.google.com/uc?id=1u98UkVheVIcyXx-ir1Gp5fggsu2V3ztC
To: C:\Users\michiel.HEPSTAR\Dropbox\Data science\Kaggle\20211206 Assessment Whereismytransport\Assessment Michiel Wolvers\data\raw\alerts.csv
100%|█████████████████████████████████████████████████████████████████████████████| 2.42M/2.42M [00:01<00:00, 1.43MB/s]
Downloading...
From: https://drive.google.com/uc?id=1KpW7s4_JfJ-rz2vT14AjL1WwPmI-3Zgy
To: C:\Users\michiel.HEPSTAR\Dropbox\Data science\Kaggle\20211206 Assessment Whereismytransport\Assessment <NAME>\data\raw\notfications.csv
100%|███████████████████████████████████████████████████████████████████████████████| 190M/190M [00:46<00:00, 4.12MB/s]
{'fraction_non_alphanumeric': 0.1947250280583614, 'fraction_numerical': 0.051627384960718295, 'mean_word_length': 7.740196078431373, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 13, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '188665', 'license': 'MIT', 'n_tokens_mistral': 844, 'n_tokens_neox': 705, 'n_words': 103, 'path': 'notebooks/0 - Download data.ipynb', 'repo_name': 'Dutchmick/Whereismytransport', 'size': '1791'}