<story><title>Implementing a Neural Network from Scratch in Python</title><url>https://victorzhou.com/blog/intro-to-neural-networks/</url></story><parent_chain><item><author>gambler</author><text>&lt;i&gt;&amp;gt;h = hidden, o = output, y_pred = predicted value of y, etc. are quite clear&lt;/i&gt;&lt;p&gt;&lt;pre&gt;&lt;code&gt;  ∀ abbr + to the cogntv load prsn wht mntn in thr hd.

  ∀ -&amp;gt; every
  + -&amp;gt; adds
  wht -&amp;gt;  who has to
  h -&amp;gt; head
&lt;/code&gt;&lt;/pre&gt;
Was that easier to read than the following?&lt;p&gt;&lt;pre&gt;&lt;code&gt;  Every abbreviation adds to the cognitive load the person has to maintain in their head.
&lt;/code&gt;&lt;/pre&gt;
This isn&amp;#x27;t code, but the principles of making code (un)readable are exactly the same.&lt;p&gt;If you see that full-word variable names create too much clutter, it means your code structure is wrong. So you fix it.&lt;p&gt;You go from this:&lt;p&gt;&lt;pre&gt;&lt;code&gt;    self.w1 = np.random.normal()
    self.w2 = np.random.normal()
    self.w3 = np.random.normal()
    self.w4 = np.random.normal()
    self.w5 = np.random.normal()
    self.w6 = np.random.normal()
&lt;/code&gt;&lt;/pre&gt;
To this:&lt;p&gt;&lt;pre&gt;&lt;code&gt;    set_to_random_numbers(self.weights)
&lt;/code&gt;&lt;/pre&gt;
After 9 years of maintaining several dozen legacy codebases (which sometimes involved financial math) I am fully convinced that the only people who like terse code are people who write it and never read it after.</text></item><item><author>jacobolus</author><text>And when trying to do algebraic manipulation on paper, having an 8 or 10 letter variable name is incredibly cumbersome.&lt;p&gt;Frankly in the middle of a numerical algorithm it is typically also cumbersome in code to have descriptive variable names for everything.&lt;p&gt;However, mathematical code (especially when written by scientists, etc.) often takes this too far, introducing many 1- or few-letter variable names without enough context or description to figure out what they stand for, and for more of the variables than necessary.&lt;p&gt;This blog post was just fine though. h = hidden, o = output, y_pred = predicted value of y, etc. are quite clear</text></item><item><author>chrisfosterelli</author><text>These are meaningful variable names in the context of a code implementation of a math algorithm. The variables directly map to the algorithmic notation.</text></item><item><author>gambler</author><text>Why are deep learning researchers allergic to meaningful variable names?</text></item></parent_chain><comment><author>chrisfosterelli</author><text>&amp;gt; Every abbreviation adds to the cognitive load the person has to maintain in their head.&lt;p&gt;I think your claim that abbreviations are always more cognitive load is wrong. For the sake of argument, let&amp;#x27;s take it at face value though. These are not even code abbreviations. They are literal translations to the math notation. The input &amp;quot;x&amp;quot; is not an abbreviation, it&amp;#x27;s a well defined value in neural network terminology.&lt;p&gt;If they called it something more descriptive like &amp;quot;input_vector_to_first_layer_of_neural_net&amp;quot;, this would be more cognitive load because someone reviewing this now needs to mentally map this to &amp;quot;x&amp;quot; in the algorithm anytime they are reviewing both.&lt;p&gt;Now, to your claim itself, I think it&amp;#x27;s unfair to say abbreviations are always more cognitive load. These variables are significantly more approachable because they follow convention. I see &amp;quot;x&amp;quot; and I know what that is. If every individual developer went ahead and rewrote every neural net with something they viewed as more interpretable in their personal context, it&amp;#x27;d be a lot harder to understand what is going on for everyone. The variable itself may be longer, but I might actually need a prose explanation of the variable&amp;#x27;s purpose because now I don&amp;#x27;t have the context of well established naming convention.</text></comment>
<story><title>Implementing a Neural Network from Scratch in Python</title><url>https://victorzhou.com/blog/intro-to-neural-networks/</url></story><parent_chain><item><author>gambler</author><text>&lt;i&gt;&amp;gt;h = hidden, o = output, y_pred = predicted value of y, etc. are quite clear&lt;/i&gt;&lt;p&gt;&lt;pre&gt;&lt;code&gt;  ∀ abbr + to the cogntv load prsn wht mntn in thr hd.

  ∀ -&amp;gt; every
  + -&amp;gt; adds
  wht -&amp;gt;  who has to
  h -&amp;gt; head
&lt;/code&gt;&lt;/pre&gt;
Was that easier to read than the following?&lt;p&gt;&lt;pre&gt;&lt;code&gt;  Every abbreviation adds to the cognitive load the person has to maintain in their head.
&lt;/code&gt;&lt;/pre&gt;
This isn&amp;#x27;t code, but the principles of making code (un)readable are exactly the same.&lt;p&gt;If you see that full-word variable names create too much clutter, it means your code structure is wrong. So you fix it.&lt;p&gt;You go from this:&lt;p&gt;&lt;pre&gt;&lt;code&gt;    self.w1 = np.random.normal()
    self.w2 = np.random.normal()
    self.w3 = np.random.normal()
    self.w4 = np.random.normal()
    self.w5 = np.random.normal()
    self.w6 = np.random.normal()
&lt;/code&gt;&lt;/pre&gt;
To this:&lt;p&gt;&lt;pre&gt;&lt;code&gt;    set_to_random_numbers(self.weights)
&lt;/code&gt;&lt;/pre&gt;
After 9 years of maintaining several dozen legacy codebases (which sometimes involved financial math) I am fully convinced that the only people who like terse code are people who write it and never read it after.</text></item><item><author>jacobolus</author><text>And when trying to do algebraic manipulation on paper, having an 8 or 10 letter variable name is incredibly cumbersome.&lt;p&gt;Frankly in the middle of a numerical algorithm it is typically also cumbersome in code to have descriptive variable names for everything.&lt;p&gt;However, mathematical code (especially when written by scientists, etc.) often takes this too far, introducing many 1- or few-letter variable names without enough context or description to figure out what they stand for, and for more of the variables than necessary.&lt;p&gt;This blog post was just fine though. h = hidden, o = output, y_pred = predicted value of y, etc. are quite clear</text></item><item><author>chrisfosterelli</author><text>These are meaningful variable names in the context of a code implementation of a math algorithm. The variables directly map to the algorithmic notation.</text></item><item><author>gambler</author><text>Why are deep learning researchers allergic to meaningful variable names?</text></item></parent_chain><comment><author>jacobolus</author><text>Are you suggesting that we should write all of our mathematical expressions using prose, and stop using symbols for operators?&lt;p&gt;That would be the original approach historically (before the past 500 years), e.g. for the quadratic formula,  Brahmagupta (628 CE):&lt;p&gt;&amp;gt; &lt;i&gt;To the absolute number multiplied by four times the square, add the square of the middle term; the square root of the same, less the middle term, being divided by twice the square is the value.&lt;/i&gt;&lt;p&gt;Go ahead and do what you like, but I doubt you’ll find many publishers who will accept your paper in the 21st century.&lt;p&gt;I know which one imposes more cognitive load for me. But disclaimer: I spent a lot of time from age 5–20 working with mathematical notation.</text></comment>