<story><title>DeepLearning10: The 8x Nvidia  GTX 1080 Ti GPU Monster</title><url>https://www.servethehome.com/deeplearning10-the-8x-nvidia-gtx-1080-ti-gpu-monster-part-1/</url></story><parent_chain></parent_chain><comment><author>highd</author><text>I&amp;#x27;ve been evaluating this space a fair bit recently. If you want to optimize FLOPS&amp;#x2F;$, especially for research-workstation sorts of setups, there&amp;#x27;s unfortunately not a lot of options for getting more than 6 GPUs in a motherboard without going to server grade where you&amp;#x27;re spending basically $3-4K more for unclear benefit - maybe a factor of two in GPU-to-GPU bandwidth.&lt;p&gt;The bitcoin miners have figured out one way to handle this, which is by using a variety of PCIe splitting systems. I&amp;#x27;ve seen examples of people putting in 8GPUs in 4 slots with these splitters. The problem is that the majority of these splitters take your x16 connection and turn into 2 to 4 x1 PCIe lanes, which is a lot of wasted bandwidth. This is fine for the miners, since the cards run mostly independently. If I could find compatible PCIe splitters that could split x16 into 2 x8 channels then that would be a really sweet spot in performance&amp;#x2F;$, but unfortunately I&amp;#x27;ve yet to find them. So right now I&amp;#x27;m going to stick to 6 GPUs, which you can get in a $500 consumer motherboard with just a few riser cables.&lt;p&gt;See for example: &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;amfeltec.com&amp;#x2F;products&amp;#x2F;flexible-x4-pci-express-4-way-splitter-gpu-oriented&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;amfeltec.com&amp;#x2F;products&amp;#x2F;flexible-x4-pci-express-4-way-s...&lt;/a&gt;</text></comment>
<story><title>DeepLearning10: The 8x Nvidia  GTX 1080 Ti GPU Monster</title><url>https://www.servethehome.com/deeplearning10-the-8x-nvidia-gtx-1080-ti-gpu-monster-part-1/</url></story><parent_chain></parent_chain><comment><author>mippie_moe</author><text>The author wasn&amp;#x27;t joking about the noise levels. This machine sounds like an F1 race car.&lt;p&gt;If you don&amp;#x27;t require a rack mounted server, a cluster of workstations like NVIDIA&amp;#x27;s DIGITS DevBox is far more cost efficient (and less noisy). I run a compute intensive business (Dreamscopeapp.com) and we opted to build a cluster of desktop-like machines instead of using a rack mounted solution. Another benefit is you don&amp;#x27;t run into the power issues mentioned in the post.&lt;p&gt;My start-up actually sells the machine described in this post:
&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;lambdal.com&amp;#x2F;nvidia-gpu-server&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;lambdal.com&amp;#x2F;nvidia-gpu-server&lt;/a&gt;&lt;p&gt;And a machine inspired by the NVIDIA DIGITS DevBox:
&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;lambdal.com&amp;#x2F;nvidia-gpu-workstation-devbox&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;lambdal.com&amp;#x2F;nvidia-gpu-workstation-devbox&lt;/a&gt;</text></comment>