<jupyter_start><jupyter_text>**Analysing Traffic Indicators**In this project, we're going to analyze a dataset about the westbound traffic on the [I-94 Interstate highway](https://en.wikipedia.org/wiki/Interstate_94).The goal of our analysis is to determine a few indicators of heavy traffic on I-94. These indicators can be weather type, time of the day, time of the week, etc.The I-94 Traffic Dataset<NAME> made the dataset available that we'll be working with, and you can download it from the [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/datasets/Metro+Interstate+Traffic+Volume).<jupyter_code>import pandas as pd, matplotlib.pyplot as plt, seaborn as sns
sns.set()
%matplotlib inline
data = pd.read_csv('Metro_Interstate_Traffic_Volume.csv', parse_dates = ['date_time'])

data.info()
data.describe()<jupyter_output><empty_output><jupyter_text>The dataset has 48,204 rows and 9 columns, and there are no null values. Each row describes traffic and weather data for a specific hour — we have data from 2012-10-02 09:00:00 until 2018-09-30 23:00:00.A station located approximately midway between Minneapolis and Saint Paul records the traffic data (see the dataset documentation). For this station, the direction of the route is westbound (i.e., cars moving from east to west). This means that the results of our analysis will be about the westbound traffic in the proximity of the station. In other words, we should avoid generalizing our results for the entire I-94 highway.<jupyter_code># Traffic volumes
plt.hist(data['traffic_volume'], bins = 20)
plt.title('Traffic Volume Distribution')
plt.show();<jupyter_output><empty_output><jupyter_text>Between 2012-10-02 09:00:00 and 2018-09-30 23:00:00, the hourly traffic volume varied from 0 to 7,280 cars, with an average of 3,260 cars.About 25% of the time, there were only 1,193 cars or fewer passing the station each hour — this probably occurs during the night, or when a road is under construction. However, about 25% of the time, the traffic volume was four times as much (4,933 cars or more).This observation gives our analysis an interesting direction: comparing daytime data with nighttime data.<jupyter_code># Day vs. night
day = data.copy()[(data['date_time'].dt.hour >= 7) 
                  & (data['date_time'].dt.hour < 19)]

night = data.copy()[(data['date_time'].dt.hour >= 19) 
                    | (data['date_time'].dt.hour < 7)]

print('Day:', day.shape[0] / len(data))
print('Night:', night.shape[0] / len(data))<jupyter_output>Day: 0.4953323375653473
Night: 0.5046676624346528<jupyter_text>This difference in row numbers between day and night is due to a few hours of missing data.<jupyter_code>plt.figure(figsize=(12,6))

# Day
plt.subplot(1, 2, 1)
plt.hist(day['traffic_volume'], bins = 20)

plt.xlim(-100, 7500)
plt.ylim(0, 8000)

plt.title('Traffic Volume: Day')
plt.ylabel('Frequency')
plt.xlabel('Traffic Volume')

# Night
plt.subplot(1, 2, 2)
plt.hist(night['traffic_volume'], bins = 20)

plt.xlim(-100, 7500)
plt.ylim(0, 8000)

plt.title('Traffic Volume: Night')
plt.ylabel('Frequency')
plt.xlabel('Traffic Volume')

plt.show();
day.describe()
night.describe()<jupyter_output><empty_output><jupyter_text>The histogram that shows the distribution of traffic volume during the day is left skewed. This means that most of the traffic volume values are high — there are 4,252 or more cars passing the station each hour 75% of the time (because 25% of values are less than 4,252).The histogram displaying the nighttime data is right skewed. This means that most of the traffic volume values are low — 75% of the time, the number of cars that passed the station each hour was less than 2,819.Although there are still measurements of over 5,000 cars per hour, the traffic at night is generally light. Our goal is to find indicators of heavy traffic, so we'll only focus on the daytime data moving forward.One of the possible indicators of heavy traffic is time. There might be more people on the road in a certain month, on a certain day, or at a certain time of day.We're going to look at a few line plots showing how the traffic volume changes according to the following:- Month- Day of the week- Time of day<jupyter_code># Time indicators
day['month'] = day['date_time'].dt.month
monthly = day.groupby('month').mean()

plt.plot(monthly['traffic_volume'])
plt.title('Volume by Hour')
plt.xlim(0, 24)
plt.show()
day['year'] = day['date_time'].dt.year
only_july = day[day['month'] == 7]
only_july.groupby('year').mean()['traffic_volume'].plot.line()
plt.show()<jupyter_output><empty_output><jupyter_text>As a tentative conclusion here, we can say that warm months generally show heavier traffic compared to cold months. In a warm month, you can can expect for each hour of daytime a traffic volume close to 5,000 cars.<jupyter_code>day['dayofweek'] = day['date_time'].dt.dayofweek
by_dayofweek = day.groupby('dayofweek').mean()
by_dayofweek['traffic_volume'].plot.line()
plt.show()<jupyter_output><empty_output><jupyter_text>Traffic volume is significantly heavier on business days (Monday – Friday). Except for Monday, we only see values over 5,000 during business days. Traffic is lighter on weekends, with values below 4,000 cars.<jupyter_code>day['hour'] = day['date_time'].dt.hour
bussiness_days = day.copy()[day['dayofweek'] <= 4] # 4 == Friday
weekend = day.copy()[day['dayofweek'] >= 5] # 5 = Saturday
by_hour_business = bussiness_days.groupby('hour').mean()
by_hour_weekend = weekend.groupby('hour').mean()


plt.figure(figsize=(11,3.5))

plt.subplot(1, 2, 1)
by_hour_business['traffic_volume'].plot.line()
plt.xlim(6,20)
plt.ylim(1500,6500)
plt.title('Traffic Volume By Hour: Monday–Friday')

plt.subplot(1, 2, 2)
by_hour_weekend['traffic_volume'].plot.line()
plt.xlim(6,20)
plt.ylim(1500,6500)
plt.title('Traffic Volume By Hour: Weekend')

plt.show()<jupyter_output><empty_output><jupyter_text>At each hour of the day, the traffic volume is generally higher during business days compared to the weekends. As somehow expected, the rush hours are around 7 and 16 — when most people travel from home to work and back. We see volumes of over 6,000 cars at rush hours.To summarise, we found a few time-related indicators of heavy traffic:- The traffic is usually heavier during warm months (March–October) compared to cold months (November–February).- The traffic is usually heavier on business days compared to weekends.- On business days, the rush hours are around 7 and 16.**Weather Indicators**Another possible indicator of heavy traffic is weather. The dataset provides us with a few useful columns about weather: temp, rain_1h, snow_1h, clouds_all, weather_main, weather_description.A few of these columns are numerical, so let's start by looking up their correlation values with traffic_volume.<jupyter_code>day.corr()['traffic_volume'].abs().sort_values(ascending = False)
day.plot.scatter('traffic_volume', 'temp', c = 'b')
plt.ylim(230, 320) # two wrong 0K temperatures mess up the y-axis
plt.show()<jupyter_output><empty_output><jupyter_text>We can conclude that temperature doesn't look like a solid indicator of heavy traffic.Let's now look at the other weather-related columns: weather_main and weather_description.<jupyter_code>by_weather_main = day.groupby('weather_main').mean()
by_weather_main['traffic_volume'].plot.barh()
plt.show()<jupyter_output><empty_output><jupyter_text>It looks like there's no weather type where traffic volume exceeds 5,000 cars. This makes finding a heavy traffic indicator more difficult. Let's also group by weather_description, which has a more granular weather classification.<jupyter_code>by_weather_description = day.groupby('weather_description').mean()
by_weather_description['traffic_volume'].plot.barh(figsize=(5,10))
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1045157988998337, 'fraction_numerical': 0.035051810157349365, 'mean_word_length': 5.6592844974446335, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '430457', 'license': 'MIT', 'n_tokens_mistral': 2619, 'n_tokens_neox': 2312, 'n_words': 891, 'path': '04. Intro to Data VIz/Traffic_Analytics.ipynb', 'repo_name': 'sciDelta/DataSci', 'size': '7821'}