<jupyter_start><jupyter_text>Read data on Airbnb property listings in Boston (source: https://www.kaggle.com/airbnb/boston)<jupyter_code>listings = pd.read_csv("listings.csv")
print(listings.shape)
listings[['id','name','neighborhood_overview']].head() # each row is a unique listing<jupyter_output>(3585, 95)<jupyter_text>What are the characteristics of each neighbourhood through the hosts' eyes?Each listing may contain the host's description of the neighbourhood his property is located in. We try to get a sense of each neighbourhood's features through the hosts' eyes by finding the most common words used in the descriptions of the homes in that neighbourhood.m<jupyter_code>nb_des = listings.copy()[['neighbourhood','neighborhood_overview']]
nb_des = nb_des[~listings.neighbourhood.isnull() & ~nb_des.neighborhood_overview.isnull()]
# for each neighbourhood, count the no. of non-null entries of neighbourhood_overview and combine them into a single string
nb_des_cnt = nb_des.groupby('neighbourhood').count()
nb_des_join = nb_des.groupby('neighbourhood')['neighborhood_overview'].apply(';'.join)
nb_des2 = pd.concat([nb_des_join, nb_des_cnt], axis=1).reset_index()
nb_des2.columns = ['neighbourhood','des_all','des_cnt']
nb_des2.head()
nb_des2<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11388455538221529, 'fraction_numerical': 0.00858034321372855, 'mean_word_length': 7.787671232876712, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '344191', 'license': 'FTL', 'n_tokens_mistral': 430, 'n_tokens_neox': 405, 'n_words': 108, 'path': '.ipynb_checkpoints/NeighbourhoodFeatures-checkpoint.ipynb', 'repo_name': 'shanghui-li/airbnb_boston', 'size': '1282'}