<story><title>New Grad vs. Senior Dev</title><url>https://ericlippert.com/2020/03/27/new-grad-vs-senior-dev/</url></story><parent_chain><item><author>corysama</author><text>It’s because Big O is Computer Science. Cache effects are Software Engineering. Professors of CS do a fine job of teaching CS. They even briefly mention that there is a implicit constant factor k in O(k n log(n)) and then they never mention it again. They certainly don’t mention that k can easily vary by 128x between algos. AKA: 7 levels of a binary tree.  Or that most of the data they will be dealing with in practice not only won’t be infinite, but will actually be less than 128 bytes.  Or, that even with huge data and an proven-ideal O() algo, there is often 10x speed-up to be had with a hybrid algo like a b-tree instead of a binary tree. And, another 2-10x with SIMD vs scalar.  100x isn’t infinite, but it’s still counts.&lt;p&gt;So, grads listen to their CS professors and that’s what they know.  It’s not until they get lectures from greybeard software engineers that they learn about the reality algos and not just the idealized algos.</text></item><item><author>dimtion</author><text>I find that the biggest misunderstanding happens because &amp;quot;new grads&amp;quot; (and I happen to be one) confuse _asymptotic complexity_ with actual complexity.&lt;p&gt;I&amp;#x27;m not sure sure why, but CS courses and interview questions mostly focus on _asymptotic complexity_ and usually forget to take into consideration the complexity for &amp;quot;little values of n&amp;quot;. And funnily enough, in real life n never goes to infinity!&lt;p&gt;In a strict sense big O notation only cares about what happens when n goes to infinity. The algorithm could behave in any way up to numbers unimaginable (like TREE(3)) but still, its big O wouldn&amp;#x27;t change.&lt;p&gt;Maybe what is missing to those &amp;quot;new grad&amp;quot; is a felling of real world data, and how a computer behave in the real world (with caches, latencies, optimised instructions etc...) not just having an ideal computer model in their mind when they design algorithms.</text></item></parent_chain><comment><author>MaxBarraclough</author><text>&amp;gt; briefly mention that there is a implicit constant factor k in O(k n log(n)) and then they never mention it again&lt;p&gt;A fine concrete example of this is the Coppersmith–Winograd algorithm (and its derivatives), a matrix multiplication algorithm with impressive complexity properties, but which in practice &lt;i&gt;always&lt;/i&gt; loses to the Strassen algorithm, despite Strassen&amp;#x27;s inferior complexity. [0][1][2]&lt;p&gt;(Aside: the Strassen algorithm is pretty mind-bending, but also easily shown. If you&amp;#x27;ve got 22 minutes spare, there&amp;#x27;s a good explanation of it on YouTube. Perhaps there&amp;#x27;s a more dense source elsewhere. [3])&lt;p&gt;&amp;gt;  It’s not until they get lectures from greybeard software engineers that they learn about the reality algos and not just the idealized algos.&lt;p&gt;To mirror what some others are saying here, students should also be taught the realities of cache behaviour, SIMD-friendliness, branch prediction, multi-threaded programming, real-time constraints, hardware acceleration, etc.&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Coppersmith%E2%80%93Winograd_algorithm&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Coppersmith%E2%80%93Winograd_a...&lt;/a&gt;&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Strassen_algorithm&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Strassen_algorithm&lt;/a&gt;&lt;p&gt;[2] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Computational_complexity_of_mathematical_operations#Matrix_algebra&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Computational_complexity_of_ma...&lt;/a&gt;&lt;p&gt;[3] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.youtube.com&amp;#x2F;watch?v=ORrM-aSNZUs&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.youtube.com&amp;#x2F;watch?v=ORrM-aSNZUs&lt;/a&gt;</text></comment>
<story><title>New Grad vs. Senior Dev</title><url>https://ericlippert.com/2020/03/27/new-grad-vs-senior-dev/</url></story><parent_chain><item><author>corysama</author><text>It’s because Big O is Computer Science. Cache effects are Software Engineering. Professors of CS do a fine job of teaching CS. They even briefly mention that there is a implicit constant factor k in O(k n log(n)) and then they never mention it again. They certainly don’t mention that k can easily vary by 128x between algos. AKA: 7 levels of a binary tree.  Or that most of the data they will be dealing with in practice not only won’t be infinite, but will actually be less than 128 bytes.  Or, that even with huge data and an proven-ideal O() algo, there is often 10x speed-up to be had with a hybrid algo like a b-tree instead of a binary tree. And, another 2-10x with SIMD vs scalar.  100x isn’t infinite, but it’s still counts.&lt;p&gt;So, grads listen to their CS professors and that’s what they know.  It’s not until they get lectures from greybeard software engineers that they learn about the reality algos and not just the idealized algos.</text></item><item><author>dimtion</author><text>I find that the biggest misunderstanding happens because &amp;quot;new grads&amp;quot; (and I happen to be one) confuse _asymptotic complexity_ with actual complexity.&lt;p&gt;I&amp;#x27;m not sure sure why, but CS courses and interview questions mostly focus on _asymptotic complexity_ and usually forget to take into consideration the complexity for &amp;quot;little values of n&amp;quot;. And funnily enough, in real life n never goes to infinity!&lt;p&gt;In a strict sense big O notation only cares about what happens when n goes to infinity. The algorithm could behave in any way up to numbers unimaginable (like TREE(3)) but still, its big O wouldn&amp;#x27;t change.&lt;p&gt;Maybe what is missing to those &amp;quot;new grad&amp;quot; is a felling of real world data, and how a computer behave in the real world (with caches, latencies, optimised instructions etc...) not just having an ideal computer model in their mind when they design algorithms.</text></item></parent_chain><comment><author>lonelappde</author><text>That&amp;#x27;s an arbitrarily restrictive view of computer science. It&amp;#x27;s like saying teaching physics ignoring friction perfectly fine.</text></comment>