<story><title>Ask HN: What maths are critical to pursuing ML/AI?</title><text>What maths must be understood to enable pursuit of either of the above fields? are there any seminal texts&amp;#x2F;courses&amp;#x2F;content which should be consumed before starting?</text></story><parent_chain><item><author>tptacek</author><text>I&amp;#x27;m not super interested in ML but I am very interested in applied mathematics in computer science. I&amp;#x27;ve got a fair bit of linear algebra due to cryptography, but have had virtually no need of any form of calculus (unless I&amp;#x27;m relying on it without knowing it) in my career.&lt;p&gt;So beyond just saying that you&amp;#x27;d need grounding in multivariable calculus to do serious ML work, I would be super interested in hearing more about why that is and what kinds of problems crop up in ML that demand it.</text></item><item><author>CuriouslyC</author><text>You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory.  It will also be helpful to be well versed in graph theory.&lt;p&gt;In my opinion one of the best starting points is &amp;quot;Information Theory, Inference and Learning Algorithms&amp;quot; by David MacKaye.  It&amp;#x27;s a bit long in the tooth now, but it is still one of the most approachable and well written books in the field.&lt;p&gt;Another old book that stands up very well is &amp;quot;Probability Theory: the Logic of Science&amp;quot; by E. T. Jaynes.&lt;p&gt;&amp;quot;Elements of Statistical Learning&amp;quot; by Tibshirani is also good.&lt;p&gt;&amp;quot;Bayesian Data Analysis&amp;quot; by Andrew Gelman is another great read.&lt;p&gt;&amp;quot;Deep Learning&amp;quot; by Ian Goodfellow and Yoshua Bengio is useful for getting caught up with recent advances in that field.</text></item></parent_chain><comment><author>tel</author><text>Calculus essentially discusses how things change smoothly and it has a very nice mechanism for talking about smooth changes algebraically.&lt;p&gt;A system which is at an optimum will, at that exact point, be no longer increasing or decreasing: a metal sheet balanced at the peak of a hill rests flat.&lt;p&gt;Many problems in ML are optimization problems: given some set of constraints, what choices of unknown parameters minimizes error? This can be very hard (NP-hard) in general, but if you design your situation to be &amp;quot;smooth&amp;quot; then you can use calculus and its very nice set of algebraic solutions.&lt;p&gt;You also need multivariate calculus because typically while you&amp;#x27;re only trying to minimize &amp;quot;error&amp;quot;, you do so by changing many, many parameters at once. This means that you&amp;#x27;ve got to talk about smooth changes in a high-dimensional space.&lt;p&gt;--&lt;p&gt;The other side of calculus is integration which talks about &amp;quot;measuring&amp;quot; how big things are. Most of probability is discussing very generalized ratios: of the total, &amp;quot;how big is &lt;i&gt;this&lt;/i&gt; piece&amp;quot; is analogous to &amp;quot;what are the odds &lt;i&gt;this&lt;/i&gt; will happen&amp;quot;.&lt;p&gt;The general discussion of measure is complex and essentially the only tool to tackle it involves gigantic (infinite, really) sums of small, well-behaved pieces to form a complex whole.&lt;p&gt;It just happens to turn out (and this is the big secret of calculus) that this machinery (integration) is dual to the study of smooth changes and you can knock them both out together.&lt;p&gt;--&lt;p&gt;So ultimately, ML hinges upon being able to measure things (integration) and talk about how they change (derivation). Those two happen to be the &lt;i&gt;same&lt;/i&gt; concept in a way and they are essentially what you study in calculus.</text></comment>
<story><title>Ask HN: What maths are critical to pursuing ML/AI?</title><text>What maths must be understood to enable pursuit of either of the above fields? are there any seminal texts&amp;#x2F;courses&amp;#x2F;content which should be consumed before starting?</text></story><parent_chain><item><author>tptacek</author><text>I&amp;#x27;m not super interested in ML but I am very interested in applied mathematics in computer science. I&amp;#x27;ve got a fair bit of linear algebra due to cryptography, but have had virtually no need of any form of calculus (unless I&amp;#x27;m relying on it without knowing it) in my career.&lt;p&gt;So beyond just saying that you&amp;#x27;d need grounding in multivariable calculus to do serious ML work, I would be super interested in hearing more about why that is and what kinds of problems crop up in ML that demand it.</text></item><item><author>CuriouslyC</author><text>You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory.  It will also be helpful to be well versed in graph theory.&lt;p&gt;In my opinion one of the best starting points is &amp;quot;Information Theory, Inference and Learning Algorithms&amp;quot; by David MacKaye.  It&amp;#x27;s a bit long in the tooth now, but it is still one of the most approachable and well written books in the field.&lt;p&gt;Another old book that stands up very well is &amp;quot;Probability Theory: the Logic of Science&amp;quot; by E. T. Jaynes.&lt;p&gt;&amp;quot;Elements of Statistical Learning&amp;quot; by Tibshirani is also good.&lt;p&gt;&amp;quot;Bayesian Data Analysis&amp;quot; by Andrew Gelman is another great read.&lt;p&gt;&amp;quot;Deep Learning&amp;quot; by Ian Goodfellow and Yoshua Bengio is useful for getting caught up with recent advances in that field.</text></item></parent_chain><comment><author>fnbr</author><text>A lot of probability theory requires it. For instance, ML is largely framed mathematically as a series of optimisation problem, which are then solved by finding the gradient and performing gradient descent; this requires elementary calculus to calculate the gradient.&lt;p&gt;Additionally, if you want to calculate a probability given a density function, or evaluate an expectation, you need to calculate several integrals. This arises quite often in the theoretical sections of ML papers&amp;#x2F;textbooks.&lt;p&gt;The use of calculus in ML is probably similar to the use of number theory in crypto- you can do applied work fine without it, but you understand the work a lot better by knowing the math, and are less likely to make dumb mistakes.</text></comment>