<jupyter_start><jupyter_text>利用Python对链家网北京主城区二手房进行数据分析* 主要讲述如何通过pandas对爬虫下来的链家数据进行相应的二手房数据分析，主要分析内容包括各个行政区，各个小区的房源信息情况。  导入链家网二手房在售房源的文件<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans

plt.rcParams['font.sans-serif'] = ['SimHei']    
plt.rcParams['axes.unicode_minus'] = False

#所有在售房源信息
house=pd.read_csv('houseinfo.csv')
house.tail(1)
house.info()
# 所有小区信息
community=pd.read_csv('community.csv')
community.tail()
# 合并小区信息和房源信息表，可以获得房源更详细的地理位置
community['community'] = community['title']
house_detail = pd.merge(house, community, on='community')
# 使数据显示完整
pd.set_option('max_columns',1000) 
pd.set_option('max_row',300) 
pd.set_option('display.float_format',lambda x:'%.5f' % x)
# 打印数据
house_detail.head(1)<jupyter_output><empty_output><jupyter_text>将数据从字符串提取出来<jupyter_code># 将字符串转换成数字
def data_adj(area_data, str):       
    if str in area_data :        
        return float(area_data[0 : area_data.find(str)])    
    else :        
        return None
# 处理房屋面积数据
house['square'] = house['square'].apply(data_adj,str = '平米')
house.head(20)
house.describe()<jupyter_output><empty_output><jupyter_text>删除车位信息<jupyter_code>car=house[house.housetype.str.contains('车位')]
# 记录中共有车位
car.shape[0]
house.drop(car.index,inplace=True)
# 现在还剩?条记录
house.shape[0]<jupyter_output><empty_output><jupyter_text>价格最高的5个别墅<jupyter_code>bieshu=house[house.housetype.str.contains('别墅')]
# 记录中共有别墅?栋
bieshu.shape[0]
bieshu.sort_values('totalPrice',ascending=False).head()<jupyter_output><empty_output><jupyter_text>删除别墅信息<jupyter_code>house.drop(bieshu.index,inplace=True)
# 现在还剩下?条记录
house.shape[0]<jupyter_output><empty_output><jupyter_text>获取总价前五的房源信息<jupyter_code>house.sort_values('totalPrice',ascending=False).head(1)<jupyter_output><empty_output><jupyter_text>获取户型数量分布信息<jupyter_code>housetype = house['housetype'].value_counts()
housetype.head(8).plot(kind='bar',x='housetype',y='size', title='户型数量分布')
plt.legend(['数量']) 
plt.show()<jupyter_output><empty_output><jupyter_text>关注人数最多5套房子<jupyter_code>house['guanzhu'] = house['followInfo'].apply(data_adj,str = '人关注')
house.sort_values('guanzhu',ascending=False).head(1)<jupyter_output><empty_output><jupyter_text>户型和关注人数分布<jupyter_code>fig, ax1 = plt.subplots(1,1)    
type_interest_group = house['guanzhu'].groupby(house['housetype']).agg([('户型', 'count'), ('关注人数', 'sum')])    
#取户型>50的数据进行可视化
ti_sort = type_interest_group[type_interest_group['户型'] > 50].sort_values(by='户型')    
ti_sort.plot(kind='barh', alpha=0.7, grid=True, ax=ax1)    
plt.title('二手房户型和关注人数分布')    
plt.ylabel('户型') 
plt.show()<jupyter_output><empty_output><jupyter_text>二手房面积分布<jupyter_code>fig,ax2 = plt.subplots(1,1)    
area_level = [0, 50, 100, 150, 200, 250, 300, 500]    
label_level = ['小于50', '50-100', '100-150', '150-200', '200-250', '250-300', '300-350']    
area_cut = pd.cut(house['square'], area_level, labels=label_level)    
# rot倾斜30°，fontsize为坐标系标记字体大小
area_cut.value_counts().plot(kind='bar', rot=30, alpha=0.4, grid=True, fontsize='small', ax=ax2)    
plt.title('二手房面积分布')    
plt.xlabel('面积')    
plt.legend(['数量'])    
plt.show()
house.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 16026 entries, 0 to 16107
Data columns (total 15 columns):
houseID       16026 non-null int64
title         16026 non-null object
link          16026 non-null object
community     16026 non-null object
years         16026 non-null object
housetype     16026 non-null object
square        16026 non-null float64
direction     16026 non-null object
floor         16026 non-null object
taxtype       15296 non-null object
totalPrice    16026 non-null int64
unitPrice     16026 non-null int64
followInfo    16026 non-null object
validdate     16026 non-null object
guanzhu       16026 non-null float64
dtypes: float64(2), int64(3), object(10)
memory usage: 2.6+ MB<jupyter_text>聚类分析<jupyter_code># 缺失值处理:直接将缺失值去掉    
cluster_data = house[['guanzhu','square','totalPrice']].dropna()    
#将簇数设为3    
K_model = KMeans(n_clusters=3)    
alg = K_model.fit(cluster_data)    
'------聚类中心------'   
center = pd.DataFrame(alg.cluster_centers_, columns=['关注人数','面积','房价'])    
cluster_data['label'] = alg.labels_ 
center
# alg.cluster_centers_
# alg.labels_ 
# cluster_data<jupyter_output><empty_output><jupyter_text>由上述整理结果可以知道，北京二手房面积低（75平米）、房价低（518万）的房源关注人数最高。从侧面也可以推断北京二手房的房价水平大多集中在518万上下。所以从营销和用户体验的角度来看，网站应该在广告和列表页的默认排序中应该给予总价518，面积75属性的二手房源更高的权重。这个类别的房源可以吸引最多的用户关注。  北京市在售面积最小二手房<jupyter_code>house.sort_values('square').iloc[0,:]<jupyter_output><empty_output><jupyter_text>北京市在售面积最大二手房<jupyter_code>house.sort_values('square',ascending=False).iloc[0,:]
house_detail.head(1)<jupyter_output><empty_output><jupyter_text>各个行政区房源均价<jupyter_code>house_unitprice_perdistrict = house_detail.groupby('district')['unitPrice'].mean()
house_unitprice_perdistrict.plot(kind='bar',rot=30,x='district',y='unitPrice', title='各个行政区房源均价')
plt.legend(['均价']) 
plt.show()<jupyter_output><empty_output><jupyter_text>各个区域房源数量排序<jupyter_code>bizcircle_count=house_detail.groupby('bizcircle').size().sort_values(ascending=False)
bizcircle_count.head(20).plot(kind='bar',x='bizcircle',y='size', title='各个区域房源数量分布')
plt.legend(['数量']) 
plt.show()<jupyter_output><empty_output><jupyter_text>各个区域均价排序<jupyter_code>bizcircle_unitprice=house_detail.groupby('bizcircle')['unitPrice'].mean().sort_values(ascending=False)
bizcircle_unitprice.head(20).plot(kind='bar',x='bizcircle',y='unitPrice', title='各个区域均价分布')
plt.legend(['均价']) 
plt.show()<jupyter_output><empty_output><jupyter_text>各个区域小区数量<jupyter_code>bizcircle_community=community.groupby('bizcircle')['title'].size().sort_values(ascending=False)
bizcircle_community.head(20).plot(kind='bar', x='bizcircle',y='size', title='各个区域小区数量分布')
plt.legend(['数量']) 
plt.show()<jupyter_output><empty_output><jupyter_text>按小区均价排序<jupyter_code>community_unitprice = house.groupby('community')['unitPrice'].mean().sort_values(ascending=False)
community_unitprice.head(15).plot(kind='bar',x='community',y='unitPrice', title='各个小区均价分布')
plt.legend(['均价']) 
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15975171512577588, 'fraction_numerical': 0.03887618425351192, 'mean_word_length': 7.964860907759883, 'pattern_counts': {'":': 0, '<': 73, '<?xml version=': 0, '>': 75, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '362577', 'license': 'Apache-2.0', 'n_tokens_mistral': 2944, 'n_tokens_neox': 2963, 'n_words': 257, 'path': 'month06/DATASCIENCE/DS_day06_student/code/.ipynb_checkpoints/lianjia-checkpoint.ipynb', 'repo_name': 'chaofan-zheng/python_learning_code', 'size': '6122'}