<jupyter_start><jupyter_text>Project 2: Kaggle Zillow prize challenge replication Due: 9:29am, Tuesday, 4 February 2020In your second project you are tasked with building a model to improve the Zestimate residual error using the data for 2017 in the Kaggle Zillow prize competition. The following description is adapted from the Kaggle. For more details, reference: https://www.kaggle.com/c/zillow-prize-1/ DataYou can only use the following data.Download the data you need for this assignment from:Collab/Resources/DatasetsThis file contains two `.csv` (comma-separated values) files, and one Excel directory file. Unzip the file to extract the CSV files into a directory of your choice. Data description(Train/Test split)- You are provided with a full list of real estate properties in three counties (Los Angeles, Orange and Ventura, California) data in 2017 in the file `properties_2017.csv`.- Not all the properties are sold in each time period. If a property was not sold in the time period, it will not have a row in `train_2017.csv` and so will not be used in predictions.(File descriptions)- properties_2017.csv - all the properties with their home features for 2017 (released on 10/2/2017)- train_2017.csv - the training set with transactions from 1/1/2017 to 9/15/2017 (released on 10/2/2017)(Data fields)- Please refer to zillow_data_dictionary.xlsx Instruction OverviewZillow Prize is challenging the data science community to help push the accuracy of the Zestimate even further. In the competition, Zillow is asking you to predict the log-error between their Zestimate and the actual sale price, given all the features of a home. The log error is defined as    logerror=log(Zestimate)−log(SalePrice)and it is recorded in the transactions file `train_2017.csv`. Using this training set and features of the home, set up your model for log error prediction. And then for each property (unique parcelid) in the `properties_2017.csv` dataset, you must predict a log error for the next period. Your program should write to output.csv in the following format.(Example)|   parcelid    |    logerror   || ------------- | ------------- ||   10754147    |    0.1234     ||   10759547    |   -0.3212     ||       ...     |       ...     |Your algorithm also needs to output the mean of all logerrors. Your answers should be in the form of a clear argument that includes both well-written prose, code and the numerical results (when the notebook is run).   AssignmentYou should complete the assignment by inserting cells in the notebook with your answers to these questions, including both prose and code you used for your analysis. Construct a model and predict the log-error for each property (unique parcelid) given all the features of a home.Let's start with the data loading.<jupyter_code># Import the libraries and give them abbreviated names:
import pandas as pd
import numpy as np
import statsmodels.api as sm

# load the data, use the directory where you saved the data:
df_properties = pd.read_csv('properties_2017.csv') 
df_train = pd.read_csv('train_2017.csv', parse_dates=["transactiondate"])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06471150781000956, 'fraction_numerical': 0.03410902135798534, 'mean_word_length': 4.169686985172982, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '119634', 'license': 'MIT', 'n_tokens_mistral': 876, 'n_tokens_neox': 774, 'n_words': 430, 'path': 'projects/project2v4.ipynb', 'repo_name': 'uvammm/uvammm.github.io', 'size': '3137'}