<story><title>FFmpeg lands CLI multi-threading as its &quot;most complex refactoring&quot; in decades</title><url>https://www.phoronix.com/news/FFmpeg-CLI-MT-Merged</url></story><parent_chain><item><author>kevincox</author><text>I&amp;#x27;ve always wondered if better multi-core performance can come from processing different keyframe segments separately.&lt;p&gt;IIUC all current encoders that support parallelism work by multiple threads working on the same frame at the same time. Often times the frame is split into regions and each thread focuses on a specific region of the frame. This approach can have a (usually small) quality&amp;#x2F;efficiency cost and requires per-encoder logic to assemble those regions into a single frame.&lt;p&gt;What if instead&amp;#x2F;additionally different keyframe segments are processed independently? So if keyframes are every 60 frames ffmpeg will read 60 frames pass that to the first thread, the next 60 to the next thread, ... then assemble the results basically by concatenating them. It seems like this could be used to parallelize any codec in a fairly generic way and it should be more efficient as there is no thread-communication overhead or splitting of the frame into regions which harms cross-region compression.&lt;p&gt;Off the top of my head I can only think of two issues:&lt;p&gt;1. Requires loading N*keyframe period frames into memory as well as the overhead memory for encoding N frames.&lt;p&gt;2. Variable keyframe support would require special support as the keyframe splits will need to be identified before passing the video to the encoding threads. This may require extra work to be performed upfront.&lt;p&gt;But both of these seem like they won&amp;#x27;t be an issue in many cases. Lots of the time I&amp;#x27;d be happy to use tons of RAM and output with a fixed keyframe interval.&lt;p&gt;Probably I would combine this with intra-frame parallelization such as process every frame with 4 threads and then run 8 keyframe segments in parallel. This way I can get really good parallelism but only minor quality loss from 4 regions rather than splitting the video into 32 regions which would harm quality more.</text></item></parent_chain><comment><author>bmurphy1976</author><text>This definitely happens.  This is how videos uploaded to Facebook or YouTube become available so quickly.  The video is split into chunks based on key frame, the chunks are farmed out to a cluster of servers and encoded in parallel, and the outputs are then re-assembled into the final file.</text></comment>
<story><title>FFmpeg lands CLI multi-threading as its &quot;most complex refactoring&quot; in decades</title><url>https://www.phoronix.com/news/FFmpeg-CLI-MT-Merged</url></story><parent_chain><item><author>kevincox</author><text>I&amp;#x27;ve always wondered if better multi-core performance can come from processing different keyframe segments separately.&lt;p&gt;IIUC all current encoders that support parallelism work by multiple threads working on the same frame at the same time. Often times the frame is split into regions and each thread focuses on a specific region of the frame. This approach can have a (usually small) quality&amp;#x2F;efficiency cost and requires per-encoder logic to assemble those regions into a single frame.&lt;p&gt;What if instead&amp;#x2F;additionally different keyframe segments are processed independently? So if keyframes are every 60 frames ffmpeg will read 60 frames pass that to the first thread, the next 60 to the next thread, ... then assemble the results basically by concatenating them. It seems like this could be used to parallelize any codec in a fairly generic way and it should be more efficient as there is no thread-communication overhead or splitting of the frame into regions which harms cross-region compression.&lt;p&gt;Off the top of my head I can only think of two issues:&lt;p&gt;1. Requires loading N*keyframe period frames into memory as well as the overhead memory for encoding N frames.&lt;p&gt;2. Variable keyframe support would require special support as the keyframe splits will need to be identified before passing the video to the encoding threads. This may require extra work to be performed upfront.&lt;p&gt;But both of these seem like they won&amp;#x27;t be an issue in many cases. Lots of the time I&amp;#x27;d be happy to use tons of RAM and output with a fixed keyframe interval.&lt;p&gt;Probably I would combine this with intra-frame parallelization such as process every frame with 4 threads and then run 8 keyframe segments in parallel. This way I can get really good parallelism but only minor quality loss from 4 regions rather than splitting the video into 32 regions which would harm quality more.</text></item></parent_chain><comment><author>cudder</author><text>I know next to nothing about video encoders, and in my naive mind I absolutely thought that parallelism would work just like you suggested it should. It sounds absolutely wild to me that they&amp;#x27;re splitting single frames into multiple segments. Merging work from different threads for every single frame sounds wasteful somehow. But I guess it works, if that&amp;#x27;s how everybody does it. TIL!</text></comment>