<story><title>A visual proof that neural nets can approximate any function</title><url>http://neuralnetworksanddeeplearning.com/chap4.html</url></story><parent_chain><item><author>a-nikolaev</author><text>Approximate, not compute. The function also must be continuous. NNs are good for approximation &amp;#x2F; interpolation &amp;#x2F; extrapolation, which makes them quite useful for certain domains of problems. But of course, it does not make them a kind of universal computing machine (in the computability sense, like universal Turing machines).</text></item></parent_chain><comment><author>13415</author><text>Wait a minute, isn&amp;#x27;t it also the case that according to the Weierstrass approximation theorem any continuous function on a closed interval can be approximated by a polynomial function? And isn&amp;#x27;t that kind of pointless for practical applications because we also need to avoid overfitting?&lt;p&gt;To clarify, I&amp;#x27;m not trying to make a snippy remark, I just happened to have used polynomial curve fitting before and looked up the Wikipedia page for the Stone-Weierstrass theorem and am trying to figure out the relevance of that post on NNs. Is it essentially the same claim?&lt;p&gt;Any clarification appreciated!</text></comment>
<story><title>A visual proof that neural nets can approximate any function</title><url>http://neuralnetworksanddeeplearning.com/chap4.html</url></story><parent_chain><item><author>a-nikolaev</author><text>Approximate, not compute. The function also must be continuous. NNs are good for approximation &amp;#x2F; interpolation &amp;#x2F; extrapolation, which makes them quite useful for certain domains of problems. But of course, it does not make them a kind of universal computing machine (in the computability sense, like universal Turing machines).</text></item></parent_chain><comment><author>electricslpnsld</author><text>&amp;gt; NNs are good for approximation &amp;#x2F; interpolation &amp;#x2F; extrapolation&lt;p&gt;Extrapolation? I was under the impression that generalizability of NNs beyond the training data was one of the major problems faced by NNs.</text></comment>