<jupyter_start><jupyter_text>To check the regression coefficients of the model<jupyter_code>sqft_model.get('coefficients')
#to explore other features in data and visalize the data
my_features=['bedrooms','bathrooms','sqft_living','sqft_lot','floors','zipcode']
sales[my_features].show()
#To see how the price varies with locality using zipcode
sales.show(view='BoxWhisker Plot',x='zipcode',y='price')
#create a new regression model with updated features found in list 'my_features'
my_features_model=graphlab.linear_regression.create(train_data,target='price',features=my_features)
# comparing  error with both previous model with one feature sqft_modl 
#and many featurs in my features model
print(sqft_model.evaluate(test_data))
print(my_features_model.evaluate(test_data))
#apply the two models to predict the prices of 3 houses 
house1=sales[sales['id']=='5309101200']<jupyter_output><empty_output><jupyter_text><jupyter_code>#to see how good our model prdict our house1 price
print('the original price of house1 is',house1['price'])
print('\nmodel using 1 feature predicts house1...\t',sqft_model.predict(house1))
print('\nmodel using multiple feature predicts house1 ...\t',my_features_model.predict(house1))

#house 2
house2=sales[sales['id']=='1925069082']
#to see how good our model prdict our house2 price
print('the original price of house2 is')
print(house2['price'])
print('\nmodel using 1 feature predicts house2...')
print(sqft_model.predict(house2))
print('\nmodel using multiple feature predicts house2 ...\t')
print(my_features_model.predict(house2))

#bill gates house
bill_gates = {'bedrooms':[8], 
              'bathrooms':[25], 
              'sqft_living':[50000], 
              'sqft_lot':[225000],
              'floors':[4], 
              'zipcode':['98039'], 
              'condition':[10], 
              'grade':[10],
              'waterfront':[1],
              'view':[4],
              'sqft_above':[37500],
              'sqft_basement':[12500],
              'yr_built':[1994],
              'yr_renovated':[2010],
              'lat':[47.627606],
              'long':[-122.242054],
              'sqft_living15':[5000],
              'sqft_lot15':[40000]}
#bill gates house using both simple model with 1 f and multi feature model
print('Multi feature model predicts bill gates house...')
print(my_features_model.predict(graphlab.SFrame(bill_gates)))
print('1 feature model predicts bill gates house')
print(sqft_model.predict(graphlab.SFrame(bill_gates)))<jupyter_output>1 feature model predicts bill gates house
[14046203.076475723]<jupyter_text>Assignment 1. Selection and summary statistics:In the notebook we covered in the module, we discovered which neighborhood (zip code) of Seattle had the highest average house sale price. Now, take the sales data, select only the houses with this zip code, and compute the average price. Save this result to answer the quiz at the end. 2. Filtering DataOne of the key features we used in our model was the number of square feet of living space (‘sqft_living’) in the house. For this part, we are going to use the idea of filtering (selecting) data.In particular, we are going to use logical filters to select rows of an SFrame. You can find more info in the Logical Filter section of this documentation.(https://turi.com/products/create/docs/generated/graphlab.SFrame.html)Using such filters, first select the houses that have ‘sqft_living’ higher than 2000 sqft but no larger than 4000 sqft. 3. Building a regression model with several more features: In the sample notebook, we built two regression models to predict house prices, one using just ‘sqft_living’ and the other one using a few more features, we called this setmy_features = ['bedrooms', 'bathrooms', 'sqft_living', 'sqft_lot', 'floors',   'zipcode']    advanced_features = ['bedrooms', 'bathrooms', 'sqft_living', 'sqft_lot', 'floors', 'zipcode','condition',  condition of house				'grade',  measure of quality of construction				'waterfront',  waterfront property				'view',  type of view				'sqft_above',  square feet above ground				'sqft_basement',  square feet in basement				'yr_built',  the year built				'yr_renovated',  the year renovated				'lat', 'long',  the lat-long of the parcel				'sqft_living15',  average sq.ft. of 15 nearest neighbors 				'sqft_lot15',  average lot size of 15 nearest neighbors ]   Create a new regression model with advanced features found in list 'advanced_features'<jupyter_code>advanced_features = [
'bedrooms','bathrooms','sqft_living','sqft_lot','floors','zipcode',
'condition','grade', 'waterfront','view','sqft_above','sqft_basement',
'yr_built','yr_renovated','lat','long','sqft_living15','sqft_lot15']
advanced_model=graphlab.linear_regression.create(train_data,target='price',features=advanced_features)<jupyter_output>PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.
          You can set ``validation_set=None`` to disable validation tracking.<jupyter_text>RMSE Error for advanced model and my features model<jupyter_code>print(my_features_model.evaluate(test_data))
print(advanced_model.evaluate(test_data))<jupyter_output>{'max_error': 3526902.937665375, 'rmse': 156847.36373149152}<jupyter_text>Number of houses in seattle alone using zipcode 98039  Question 1:Selection and summary statistics: We found the zip code with the highest average house price. What is the average house price of that zip code?<jupyter_code>import graphlab.aggregate as agg
avg_zp=sales.groupby(key_columns=['zipcode'],operations={'AvgHousePrice':agg.MEAN('price')})
max_zp=avg_zp[avg_zp['AvgHousePrice']==avg_zp['AvgHousePrice'].max()]
print('The highest average price of house is found in zipcode.. ')
print(max_zp['zipcode'])
print('And the that average price is....')
print(max_zp['AvgHousePrice'])

#Method 2

seattle_data=sales[sales['zipcode']=='98039']
#seattle_data.show()
answer1=seattle_data['price'].mean()
print('The mean average price of a house in the seattle by method 2 is ',answer1)<jupyter_output>The highest average price of house is found in zipcode.. 
['98039', ... ]
And the that average price is....
[2160606.6000000006, ... ]
('The mean average price of a house in the seattle by method 2 is ', 2160606.5999999996)<jupyter_text>Question 2:Filtering data: What fraction of the houses have living space between 2000 sq.ft. and 4000 sq.ft.?<jupyter_code>filteredHouses=sales[(sales['sqft_living']>2000) & (sales['sqft_living']<4000)]
x=float(filteredHouses.num_rows())
y=float(sales.num_rows())
#print(x,y,x/y,type(x),type(y))

print('The fraction of houses between 2000 sq.ft and 4000 sq.ft is',x/y)<jupyter_output>('The fraction of houses between 2000 sq.ft and 4000 sq.ft is', 0.4215518437977143)<jupyter_text>Question 3:Building a regression model with several more features: What is the difference in RMSE between the model trained with my_features and the one trained with advanced_features?<jupyter_code>advanced_features = [
'bedrooms','bathrooms','sqft_living','sqft_lot','floors','zipcode',
'condition','grade', 'waterfront','view','sqft_above','sqft_basement',
'yr_built','yr_renovated','lat','long','sqft_living15','sqft_lot15']
#sales[advanced_features].show()<jupyter_output><empty_output><jupyter_text>Difference of RMSE between the advanced model and my features model<jupyter_code>a=my_features_model.evaluate(test_data)
RMSE_myFeatures=a['rmse']
b=advanced_model.evaluate(test_data)
RMSE_advFeatures=b['rmse']
answer3=RMSE_advFeatures-RMSE_myFeatures
print('Difference of RMSE between the advanced model and my features model is',answer3)
print('Therefore the RMSE of Advanced model is less than 22685$ compared to RMSE of my_features model')<jupyter_output>Therefore the RMSE of Advanced model is less than 22685$ compared to RMSE of my_features model<jupyter_text>Model Visualizationvisualizing how our prdictions look like by plotting the predicted prices for test data using 1 feature model,Multi feature model and advanced feature model with actual test prices of the test data<jupyter_code>import matplotlib.pyplot as plt
%matplotlib inline
plt.scatter(test_data['sqft_living'],test_data['price'],label='Actual price',marker='.',color='blue')
plt.scatter(test_data['sqft_living'],sqft_model.predict(test_data),label='one feature model',
            marker='*',color='magenta')

plt.legend()
plt.title('Comparison of actual and simple ML model by Srinivas')
         
plt.scatter(test_data['sqft_living'],test_data['price'],label='Actual price',marker='.',color='darkblue')
plt.scatter(test_data['sqft_living'],my_features_model.predict(test_data),label='multi feature model',
    marker='+', color='cyan')
plt.legend()
plt.title('Comparison of actual and multi features model by Srinivas')
plt.scatter(test_data['sqft_living'],test_data['price'],label='Actual price',marker='.',color='blue')
plt.scatter(test_data['sqft_living'],advanced_model.predict(test_data),label='advanced feature model',
        marker='*',color='red')
plt.legend()
plt.title('Comparison actual price and advanced model by Srinivas')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11695585328354931, 'fraction_numerical': 0.034482758620689655, 'mean_word_length': 5.883597883597884, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '629463', 'license': 'MIT', 'n_tokens_mistral': 3091, 'n_tokens_neox': 2741, 'n_words': 822, 'path': 'Predicting  Housing prices through ML regression.ipynb', 'repo_name': 'srinivasgln/Selected-Projects', 'size': '9106'}