<story><title>A new type of neural network is more interpretable</title><url>https://spectrum.ieee.org/kan-neural-network</url></story><parent_chain></parent_chain><comment><author>Ameo</author><text>I&amp;#x27;ve tried out and written about[1] KANs on some small-scale modeling, comparing them to vanilla neural networks, as previously discussed here: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=40855028&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=40855028&lt;/a&gt;.&lt;p&gt;My main finding was that KANs are very tricky to train compared to NNs. It&amp;#x27;s usually possible to get per-parameter loss roughly on par with NNs, but it requires a lot of hyperparameter tuning and extra tricks in the KAN architecture. In comparison, vanilla NNs were much easier to train and worked well under a much broader set of conditions.&lt;p&gt;Some people commented that we&amp;#x27;ve invested an incredible amount of effort into getting really good at training NNs efficiently, and many of the things in ML libraries (optimizers like Adam, for example) are designed and optimized specifically for NNs. For that reason, it&amp;#x27;s not really a good apples-to-apples comparison.&lt;p&gt;I think there&amp;#x27;s definitely potential in KANs, but they aren&amp;#x27;t a magic bullet. I&amp;#x27;m also a bit dubious about interpretability claims; the splines that are usually used for KANs don&amp;#x27;t really offer much more insight to me than just analyzing the output of a neuron in a lower layer of a NN.&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;cprimozic.net&amp;#x2F;blog&amp;#x2F;trying-out-kans&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;cprimozic.net&amp;#x2F;blog&amp;#x2F;trying-out-kans&amp;#x2F;&lt;/a&gt;</text></comment>
<story><title>A new type of neural network is more interpretable</title><url>https://spectrum.ieee.org/kan-neural-network</url></story><parent_chain></parent_chain><comment><author>yorwba</author><text>Previous discussion of Kolmogorov-Arnold networks: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=40219205&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=40219205&lt;/a&gt;</text></comment>