<story><title>Generalized linear models, abridged</title><url>http://bwlewis.github.io/GLM/</url></story><parent_chain></parent_chain><comment><author>imurray</author><text>If you mainly care about prediction, rather than inspecting the fitted parameters, a lot of this detail is usually overkill.&lt;p&gt;To generalize well, it&amp;#x27;s almost always a good idea to have some sort of regularization, such as penalizing the sum of the square of the parameters. The extra term in the cost function will usually make the naive &amp;quot;normal equations&amp;quot; approach work fine, and give much the same predictions as fancy pivoted QR approaches. On my machine it&amp;#x27;s also a lot faster (the ball-park is ~~10x faster for large systems).&lt;p&gt;I&amp;#x27;m glad R has super-solid robust GLM implementations. And unless you&amp;#x27;re fitting &lt;i&gt;many&lt;/i&gt; models, you should probably just use such a library routine. However, I wish more tutorials and textbooks would spend more time on the reasons for numerical stability, and when one should care, rather than pushing that detail off into a trail of citations.</text></comment>
<story><title>Generalized linear models, abridged</title><url>http://bwlewis.github.io/GLM/</url></story><parent_chain></parent_chain><comment><author>AtheMathmo</author><text>This is a great post! I tried implementing my own GLMs [1] a short while ago. But I ran into a lot of trouble with numerical instability and had a hard time tracking down ways to solve these edge cases.&lt;p&gt;Hopefully with this as a resource I&amp;#x27;ll be able to make some more progress on it!&lt;p&gt;[1]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;AtheMathmo&amp;#x2F;rusty-machine&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;src&amp;#x2F;learning&amp;#x2F;glm.rs&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;AtheMathmo&amp;#x2F;rusty-machine&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;src&amp;#x2F;...&lt;/a&gt;</text></comment>