Comparing the approaches
Machine Learning: Regression
Curso 2 de 4 en SpecializationMachine Learning
Case Study - Predicting Housing Prices In our first case study, predicting house prices, you will create models that predict a continuous value (price) from input features (square footage, number of bedrooms and bathrooms,...). This is just one of the many places where regression can be applied. Other applications range from predicting health outcomes in medicine, stock prices in finance, and power usage in high-performance computing, to analyzing which regulators are important for gene expression. In this course, you will explore regularized linear regression models for the task of prediction and feature selection. You will be able to handle very large sets of features and select between models of various complexity. You will also analyze the impact of aspects of your data -- such as outliers -- on your selected models and predictions. To fit these models, you will implement optimization algorithms that scale to large datasets. Learning Outcomes: By the end of this course, you will be able to: -Describe the input and output of a regression model. -Compare and contrast bias and variance when modeling data. -Estimate model parameters using optimization algorithms. -Tune parameters with cross validation. -Analyze the performance of the model. -Describe the notion of sparsity and how LASSO leads to sparse solutions. -Deploy methods to select between models. -Exploit the model to form predictions. -Build a regression model to predict prices using a housing dataset. -Implement these techniques in Python.
Our course starts from the most basic regression model: Just fitting a line to data. This simple model for forming predictions from a single, univariate feature of the data is appropriately called "simple linear regression".<p> In this module, we describe the high-level regression task and then specialize these concepts to the simple linear regression case. You will learn how to formulate a simple regression model and fit the model to data using both a closed-form solution as well as an iterative optimization algorithm called gradient descent. Based on this fitted function, you will interpret the estimated model parameters and form predictions. You will also analyze the sensitivity of your fit to outlying observations.<p> You will examine all of these concepts in the context of a case study of predicting house prices from the square feet of the house.
Computing the gradient of RSS7:18
Approach 1: closed-form solution5:33
Approach 2: gradient descent7:12
Comparing the approaches1:38
Okay, so let's take a moment to compare the two approaches that we've gone over,
either setting the gradient equal to zero or doing gradient descent.
Well, in the case of minimizing residual sum of squares,
we showed that both were fairly straight forward to do.
But in a lot of the machine learning method's that we're interested in
taking the gradient and setting it equal to zero,
well there's just no close form solution to that problem.
So, often we have to turn to method's like gradient descent.
And likewise, as we're gonna see in the next module, where we turn to having lots
of different inputs, lots of different features in our regression.
Even though there might be a close form solution to setting the gradient equal to
zero, sometimes in practice it can be much more efficient
computationally to implement the gradient descent approach.
And finally one thing that I should mention about the gradient descent
approach is the fact that in that case, we had to choose a stepsize and
a convergence criteria.
Whereas, of course, if we take the gradient and
are able to set it to zero, we don't have to make any of those choices.
So that is a downside to the gradient descent approach,
is having to specify these parameters of the algorithm.
But sometimes we're relying on these types of optimization algorithms to