<jupyter_start><jupyter_text>In this notebook, we'll take a look at how Airbnb prices in NYC have trended during COVID-19 from Aug 2020 to Aug 2021.<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import os
import glob
import time
# read listings csv for each individual month into one dataframe
# create a new column 'date' by extracting the month and year from each file name
filenames = glob.glob(os.path.join('data', 'listings_20*.csv'))
df = pd.concat(pd.read_csv(file).assign(date=file[-10:-4]) for file in filenames)
df.head()
df.shape
len(df.id.unique())<jupyter_output><empty_output><jupyter_text>There are 62,628 unique properties during this 12-month period. (Data is scraped at the beginning of each month so we effectively have data from Aug 2020 - Jul 2021)Let's take a look at average price over time across the entire city.<jupyter_code># reformat date column to be more readable
df.date = df.date.apply(lambda x:x[:4] + '-' + x[4:])
df.groupby('date')['price'].agg(['mean', 'median'])<jupyter_output><empty_output><jupyter_text>It looks like the average price as measured both by the mean and median decreased from end of 2020 to 1Q 2021 and then increased again. Let's visualize both the mean and the distribution of prices by month to see the trend more clearly.<jupyter_code>df.groupby('date')['price'].mean().plot()
plt.ylabel('Average Airbnb price in NYC')
plt.xlabel('Month')
plt.savefig('figures/price_over_time', dpi=200)
date_order = sorted(df.date.unique())
plt.figure(figsize=(6,4), dpi=80)
sns.boxplot(x='date', y='price', data=df, showfliers=False, order=date_order)
plt.xticks(rotation=40)<jupyter_output><empty_output><jupyter_text>The distribution of listing prices looks pretty similar from month to month, and we can see the slight increase of prices at the start of summer 2021. Next, let's take a look at the trends by borough.<jupyter_code>avgprice_by_borough
avgprice_by_borough = df.groupby(['date', 'neighbourhood_group']).agg(mean_price=('price', 'mean'))
boroughs = ['Manhattan', 'Brooklyn', 'Queens', 'Bronx', 'Staten Island']
avgprice_by_borough = avgprice_by_borough.reindex(boroughs, level='neighbourhood_group') # re-order boroughs to match plots in price_prediction.ipynb
sns.lineplot(data=avgprice_by_borough, x='date', y='mean_price', hue='neighbourhood_group')
plt.legend(bbox_to_anchor=(1.05, 0.6), loc=2)
plt.ylabel('Average Airbnb price')
plt.xlabel('Month')
plt.xticks(rotation=40)
plt.savefig('figures/price_over_time_borough', dpi=200)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11855869817900039, 'fraction_numerical': 0.025184037194885704, 'mean_word_length': 6.7074626865671645, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '462715', 'license': 'CC0-1.0', 'n_tokens_mistral': 917, 'n_tokens_neox': 834, 'n_words': 265, 'path': '.ipynb_checkpoints/price_trends-checkpoint.ipynb', 'repo_name': 'jaymeszm/airbnb', 'size': '2581'}