<story><title>The compositor is evil</title><url>https://raphlinus.github.io/ui/graphics/2020/09/13/compositor-is-evil.html</url></story><parent_chain></parent_chain><comment><author>PaulDavisThe1st</author><text>It is fascinating to me, as someone who has been doing real time audio work for 20 years, to see how two fundamental processes have been at work during that time:&lt;p&gt;&lt;pre&gt;&lt;code&gt;   * audio developers recognizing the parallels between the way audio and video hardware works, and taking as many lessons from the video world as possible. The result is that audio data flow (ignoring the monstrosity of USB audio) is now generally much better than it used to be, and in many cases is conceptually very clean. In addition, less and less processing is done by audio hardware, and more and more by the CPU.

   * video developers not understanding much about audio, and failing to notice the parallels between the two data handling processes. Rather than take any lessons from the audio side of things, stuff has just become more and more complex and more and more distant from what is actually happening in hardware. In addition, more and more processing is done by video hardware, and less and less by the CPU.
&lt;/code&gt;&lt;/pre&gt;
In both cases, there is hardware which requires data periodically (on the order of a few msec). There are similar requirements to allow multiple applications to contribute what is visible&amp;#x2F;audible to the user. There are similar needs for processing pipelines between hardware and user space code.&lt;p&gt;(one important difference: if you don&amp;#x27;t provide a new frame of video data, most humans will not notice the result; if you don&amp;#x27;t provide a new frame of audio fata, every human will hear the result).&lt;p&gt;I feel as if both these worlds would really benefit from somehow having a full exchange about the high and low level aspects of the problems they both face, how they have been solved to date, how might be solved in the future, and how the two are both very much alike, and really quite different.</text></comment>
<story><title>The compositor is evil</title><url>https://raphlinus.github.io/ui/graphics/2020/09/13/compositor-is-evil.html</url></story><parent_chain></parent_chain><comment><author>monocasa</author><text>I&amp;#x27;ve always really liked the Nintendo DS&amp;#x27;s GPU, and thought something with a similar architecture but scaled up would make a lot sense in a few places in the stack.  Unlike most GPUs, you submitted a static scene, and then the rendering happened pretty close to lock step with the scan out to the LCD.  There was a buffer with a handful of lines stored, but it was pretty close; the latency was pretty much unbeatable.  In a lot fo ways it was really a 3D extension to their earlier 2D (NES, SNES, GB, GBA) GPU designs.&lt;p&gt;Something like that sitting where the scan out engines exist today (with their multiple planes the composite today) would be absolutely killer if you could do hardware software co design to take advantage of it.&lt;p&gt;I&amp;#x27;ve also thought that something like that would be great in VR&amp;#x2F;AR.</text></comment>