<story><title>I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]</title><url>https://penberg.org/parakernel-hotos19.pdf</url></story><parent_chain><item><author>thaumaturgy</author><text>The paper reads like it&amp;#x27;s suggesting moving the burden of complexity in dealing with varying hardware interfaces from the kernel to userland so that userland can take direct advantage of higher performance hardware when it&amp;#x27;s available.&lt;p&gt;I could see that for some very small niches, but in general I think it would be a terrible development for the industry.&lt;p&gt;Hardware vendors don&amp;#x27;t like to share. They don&amp;#x27;t share code, they don&amp;#x27;t share common interfaces, they don&amp;#x27;t even share documentation. As it is now, these are all problems which most userland developers don&amp;#x27;t have to care about -- those problems get dealt with in the kernel, by developers who specialize in building support for uncooperative hardware.&lt;p&gt;The average application developer doesn&amp;#x27;t want to have to figure out how many queues are supported by a NIC just to open a connection on the network. Further: the average application developer isn&amp;#x27;t experienced enough to do this correctly.&lt;p&gt;Given the niche where these tradeoffs make sense, I&amp;#x27;m not sure why the paper bothers to emphasize security at all.</text></item></parent_chain><comment><author>saltcured</author><text>There is a constant dance at the fringe of high performance systems. It leads to a recurring pattern of &amp;quot;revolutionizing&amp;quot; with some kind of bypass or coprocessor architecture, then eventually reverting to traditional structures as the new performance realities reach commodity levels.&lt;p&gt;Part of it is the economics at the fringe can pursue speed at any cost. And part is the heady appeal of doing things differently for researchers and advanced practitioners. But, in the long view, I think you are right that it is a bad idea. If you care about maintenance and sustainability, you usually find that these bypass solutions get abandoned as soon as the more conventional approaches can approximate their speed on newer commodity hardware. So there is huge churn in these specialist devices with specialist APIs and tooling.&lt;p&gt;There is a recurring theme in high performance networking where crazy things are tried and all sorts of fancy protocol offloading written, then eventually deprecated because it is seen as a support burden and a source of bugs. Because each of these specialized stacks has a smaller user base, they are have less economy of scale to invest in maintenance and stabilization.</text></comment>
<story><title>I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]</title><url>https://penberg.org/parakernel-hotos19.pdf</url></story><parent_chain><item><author>thaumaturgy</author><text>The paper reads like it&amp;#x27;s suggesting moving the burden of complexity in dealing with varying hardware interfaces from the kernel to userland so that userland can take direct advantage of higher performance hardware when it&amp;#x27;s available.&lt;p&gt;I could see that for some very small niches, but in general I think it would be a terrible development for the industry.&lt;p&gt;Hardware vendors don&amp;#x27;t like to share. They don&amp;#x27;t share code, they don&amp;#x27;t share common interfaces, they don&amp;#x27;t even share documentation. As it is now, these are all problems which most userland developers don&amp;#x27;t have to care about -- those problems get dealt with in the kernel, by developers who specialize in building support for uncooperative hardware.&lt;p&gt;The average application developer doesn&amp;#x27;t want to have to figure out how many queues are supported by a NIC just to open a connection on the network. Further: the average application developer isn&amp;#x27;t experienced enough to do this correctly.&lt;p&gt;Given the niche where these tradeoffs make sense, I&amp;#x27;m not sure why the paper bothers to emphasize security at all.</text></item></parent_chain><comment><author>josephg</author><text>I don’t think anyone is suggesting that every application explicitly code for each network device. All of that tricky logic could be put into a userland library instead of the kernel. If we wanted, we could even replicate a kernel device driver style API in userland that network device drivers program against, done as a set of userland library files loaded dynamically based on detected hardware.&lt;p&gt;The tricky part wouldn’t be sharing code between applications.  We know how to do that. The hard part would be figuring out a clean way to share the hardware between all running applications, given that any app could be terminated at any time, apps might be mutually untrustworthy and apps would have to play nice to share resources. I can imagine a hybrid approach where the kernel allocates network queues to applications and suggests userland device drivers. While running, the apps would have direct access to the hardware. And when the app is terminated the kernel would reclaim the assigned hardware for reuse by other applications.</text></comment>