<story><title>Cerebras’s giant chip will smash deep learning’s speed barrier</title><url>https://spectrum.ieee.org/semiconductors/processors/cerebrass-giant-chip-will-smash-deep-learnings-speed-barrier</url></story><parent_chain></parent_chain><comment><author>geomark</author><text>The article talks about a few things that they call inventions, like making interconnections across what would normally be scribe lines. But I personally worked on wafer scale integration about 25 years ago and we were already doing that. We called it inter-reticle stitching. The technology was ancient back then - 0.5 micron feature size on 4 inch wafers - but the wafer scale techniques are applicable to modern technologies. In particular, developing a yield model that informs your on-chip redundancy choices and designing built-in self test and selection circuitry so that you can yield large chips. The chip we developed was so large that only two would fit on a wafer. We got 50% yield on a line that was far from mature at the time. The company lacked the vision to do anything with what they had developed. To them it was just a chip for which there were few customers. The suits didn&amp;#x27;t know how to make bank with this methodology that could yield nearly arbitrarily complex chips in nearly any target process.&lt;p&gt;Edit: There were a number of papers and conference proceedings published back then but not much shows up when searching Google. Here&amp;#x27;s one discussing the issues and results of field stitching &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;fdocuments.in&amp;#x2F;document&amp;#x2F;ieee-comput-soc-press-1992-international-conference-on-wafer-scale-integration-589dfe172703a.html&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;fdocuments.in&amp;#x2F;document&amp;#x2F;ieee-comput-soc-press-1992-in...&lt;/a&gt;&lt;p&gt;From 1992, so yeah, field stitching is not a recent invention.</text></comment>
<story><title>Cerebras’s giant chip will smash deep learning’s speed barrier</title><url>https://spectrum.ieee.org/semiconductors/processors/cerebrass-giant-chip-will-smash-deep-learnings-speed-barrier</url></story><parent_chain></parent_chain><comment><author>mark_l_watson</author><text>I don’t know if this mega-chip will be successful, but I like the idea. Before I retired I managed a deep learning team that had a very cool internal product for running distributed TensorFlow. Now in retirement I get by with a single 1070 GPU for experiments - not bad but having something much cheaper, much more memory, and much faster would help so much.&lt;p&gt;I tend to be optimistic, so take my prediction with a grain of salt: I bet within 7 or 8 years there will be an inexpensive device that will blow away what we have now. There are so many applications for much larger end to end models that will but pressure on the market for something much better than what we have now. BTW, the ability to efficiently run models on my new iPhone 11 Pro is impressive and I have to wonder if the market for super fast hardware for training models might match the smartphone market. For this to happen, we need a deep learning rules the world shift. BTW, off topic, but I don’t think deep learning gets us to AGI.</text></comment>