<jupyter_start><jupyter_text>Data Science Intensive Course Capstone project: Flight Delays (Brazil)The microdata from ANAC [(National Agency for Civil Aviation)](http://www.anac.gov.br/assuntos/dados-e-estatisticas/historico-de-voos) contains all flights that departure from brazilian airports. The flights are grouped by year-month, from January, 2000 to October, 2017 (last access on November, 2017).According to this local publication, in Brazil, it is estimated that 20% of the 700K flights/year have some kind of delay: [http://infograficos.oglobo.globo.com/economia/raio-x-dos-atrasos-dos-voos.html](http://infograficos.oglobo.globo.com/economia/raio-x-dos-atrasos-dos-voos.html)---<jupyter_code># Pandas, Numpy, Matplotlib 
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

## plt.style.use('ggplot')<jupyter_output><empty_output><jupyter_text>The raw data (monthly microdata from ANAC)I will use the microdata that contains flights that occured on **aug, 2017**:* **Flights**: 94,185* **Flights with delay**: 16,317 (17,32%)---  The questions:  1. It is possible to forecast flights with delay based on past delays causes? 2. Which model to choose?--- ![Machine Learning Models](../images/machine-learning-600x429.png)<jupyter_code># Flights (aug, 2017)
flightsAug17 = 'data/flightAug2017.csv'
flights = pd.read_csv(flightsAug17, sep = ';', parse_dates = True)
flights.describe()
flights.head(5)
flights[flights['DelayMotivationCode'] >'NaN'].describe()
flights[flights['DelayMotivationCode'] >'NaN'].head(5)
# Total of delay per motivation code
flights['DelayMotivationCode'].value_counts()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1251497005988024, 'fraction_numerical': 0.03413173652694611, 'mean_word_length': 6.110638297872341, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '339936', 'license': 'Apache-2.0', 'n_tokens_mistral': 610, 'n_tokens_neox': 535, 'n_words': 153, 'path': 'notebooks/flightdelays.ipynb', 'repo_name': 'davicsilva/dsintensive', 'size': '1670'}