<story><title>Linux Pipes Are Slow</title><url>https://qsantos.fr/2024/08/25/linux-pipes-are-slow/</url></story><parent_chain><item><author>AkBKukU</author><text>I have a project that uses a proprietary SDK for decoding raw video. I output the decoded data as pure RGBA in a way FFMpeg can read through a pipe to re-encode the video to a standard codec. FFMpeg can&amp;#x27;t include the Non-Free SDK in their source, and it would be wildly impracticable to store the pure RGBA in a file. So pipes are the only way to do it, there are valid reasons to use high throughput pipes.</text></item><item><author>0xbadcafebee</author><text>Calling Linux pipes &amp;quot;slow&amp;quot; is like calling a Toyota Corolla &amp;quot;slow&amp;quot;. It&amp;#x27;s fast enough for all but the most extreme use cases. Are you racing cars? In a sport where speed is more important than technique? Then get a faster car. Otherwise stick to the Corolla.</text></item></parent_chain><comment><author>ploxiln</author><text>What percentage of CPU time is used by the pipe in this scenario? If pipes were 10x faster, would you really notice any difference in wall-clock-time or overall-cpu-usage, while this decoding SDK is generating the raw data and ffmpeg is processing it? Are these video processing steps anywhere near memory copy speeds?</text></comment>
<story><title>Linux Pipes Are Slow</title><url>https://qsantos.fr/2024/08/25/linux-pipes-are-slow/</url></story><parent_chain><item><author>AkBKukU</author><text>I have a project that uses a proprietary SDK for decoding raw video. I output the decoded data as pure RGBA in a way FFMpeg can read through a pipe to re-encode the video to a standard codec. FFMpeg can&amp;#x27;t include the Non-Free SDK in their source, and it would be wildly impracticable to store the pure RGBA in a file. So pipes are the only way to do it, there are valid reasons to use high throughput pipes.</text></item><item><author>0xbadcafebee</author><text>Calling Linux pipes &amp;quot;slow&amp;quot; is like calling a Toyota Corolla &amp;quot;slow&amp;quot;. It&amp;#x27;s fast enough for all but the most extreme use cases. Are you racing cars? In a sport where speed is more important than technique? Then get a faster car. Otherwise stick to the Corolla.</text></item></parent_chain><comment><author>CyberDildonics</author><text>&lt;i&gt;So pipes are the only way to do it&lt;/i&gt;&lt;p&gt;Lets not get carried away. You can use ffmpeg as a library and encode buffers in a few dozen lines of C++.</text></comment>