Modularity & Generational Core The future of Fedora?
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Šabata, Petr
Samalik, Adam
Following the progress of the Modularity initiative and Factory 2.0developments in Fedora, it's about time we define what the next, modularoperating system should look like, focusing on the developer’s point of view. This first half of this talk will present the main concepts of Modularity,explaining in detail what modules are, how they get built, composed,distributed and kept updated once deployed on the target system. The second half of the talk will explore the idea of the so-calledGenerational Core, often also referred to as Base Runtime — a modular platformbase on top of which application modules can both be built and executed. Both these topics cover ongoing efforts and we don’t yet have definitiveanswers to all the questions. We’ll talk about what we learned in the pastyear, what problems we’re trying to solve and our plans for the future.
 So, hi everyone. This is Peter. My name is Adam. We are from Red Hat and we are working on a project called Fedora modularity. By the way, we have stickers for you. So, if you come here after the talk, you can get one or two. Only 25 of them. Yeah, only 25, but that's fine. So, I'm not sure who of you was on Languines Talk last year, here on FOSDEM. He was talking about how distributions are great, but not really, and how we might need to change them to be ready for the future. So, let's start with why they are great, because I think that's still true. They ship package software, which means it's pretty easy to install. They also ship dependencies and everything else with it. So, it just works. They integrate it and test it. So, nothing breaks, really. And some important thing. It's also patched for security vulnerabilities, because, for example, if you are a developer and you're developing your application, which is included in the distribution, you might not have pretty much time to patch for some security holes. So, distributions can do it for you. And distos have a lifespan, which basically means they're released at some point, die at some point. This is, for example, Fedora. And if we have a look at a bit more detail, we can see that I have, for example, distribution version one. It includes some libraries and software, which is pretty much frozen for the whole lifetime. And then I have another distribution, another version, and the same story there. This is fine, but there are some problems, because, for example, applications are released at their own pace, independent of distributions. And they can also come in multiple versions, which means conflicts, right? Distributions often include only one version of every application, because they got dependencies, they can conflict with each other, and it would be impossible to install. So, we need to choose a version, but how to choose the right one? That's a tricky question, because, for example, if you are a developer, you might prefer this one, fast and latest, because that's what developers want. But if you're a sysadmin, for example, you're running an application, you don't want it to change so much, right? So, you might prefer the other scenario. So, I guess the right answer, we need all of them, we can't just choose one. But how to do it? Well, there are several solutions to this. So, for example, we can have distribution on a fast track and another one, which is a bit slower. So, for example, this can be Fedora, this can be CentOS, or Ubuntu LTS. If I'm a bit more crazy, maybe I can have continuous upgrades. But this still doesn't allow me to mix and match versions as I want. So, another solution is a software collection. How many of you heard about software collections? All right, so that's like half. So, software collections are kind of special packages, which entail software in a separate space. So, I can install more versions of the software on the system. But it's pretty hacky, it doesn't work always. So, that's a solution but not ideal. Another thing would be Linux containers. Who of you know Linux containers? Almost everyone, that's great. So, yeah, we know it's an isolated space, which looks like a system, and it's for single application. And I can multiple containers on system. But sometimes, if I put stuff in the container, I need to look after for it. I might have problems with updates. And still, I'm building containers out of a single distribution. So, if I install some different version of something, this can get conflicted as well. So, that's why we are working on modularity, which kind of combines everything together. And now I have like one minute short animation with subtitles, so I won't be talking. So, if you could please read. Or you can just read the subtitles. No. This is a talk, I'm not supposed to be talking here. All right, so that's basically it. So, it will look like this. We will have something called Base Runtime, which is a tiny system with a pretty long life cycle. And another software, and the software will be like, how do you call it in English? Independent, sorry, independent of the distribution, right? So, as I said, Base Runtime, this is what Peter is going to talk about, which is a small system containing software with proven API stability, like kernel and glibc, and some basic tools to make it boot. And the software run as a module, and module is similar concept to Android applications or iPhone applications. So, it's pretty independent of the system, and it includes all the dependencies in itself. And as I said, modules are like applications, so it can be database, it can be Firefox, but it can be also LAMP stack or basically any group of packages. And it will look like this. So, I will have packages and some metadata, and I can also have packages as an API and as dependencies, which the difference is that if I, for example, have a Firefox module, the Firefox package will be API. This is the only package I care for, but I can have all of the dependencies, libraries, et cetera, which are not guaranteed to be stable. But users won't care about this, users care only about the API. And modules are defined as a bi-module-md file, and it basically defines everything, which I'll show you right now. So, this is the module-md. It's kind of cut off on the top, but there's nothing important. Important is that we have some names to inversion, which is an ID of the module. Description, licensing, some references, where the project leaves, backtracking, some optional metadata, and here is the important part. So, I have dependencies, there is the base runtime, and to build it, there are build-requires, which is base runtime and base runtime build. I guess Peter will tell you more about it. And these are lists of the components. So, I can include packages in my module, and I can include another modules in my module, which might be useful. For example, if I have an application requiring Python, I don't have to list all the Python packages, I just list the Python module. And in the RBM world, if I have a source package, I can build multiple binary packages of it, but... Oh, sorry. So, in the RBM world, I have source RBM packages, which are packages containing the source code, and if I build it, I can build multiple binaries. And I don't want to probably include all the binaries. There can be devl packages and stuff, so I can filter them. Thanks to this filter, here I can specify the RPMs API, and in the future, there will be more types of API, not just RPMs, and also something called install profile, which is interesting, basically explains how to install the module, and I can have more ways to installing. So, for example, if I have web server, I can install it as a production or as a developer with a bit different configuration. So, that was module MD, and module MD is... Modules are built in something called factory 2.0. I don't have time to talk about factory 2.0, but if you can Google, for example, or find talk recordings from DevCon, which was like last week, you can find some interesting stuff about that. And when we build them, modules are delivered as multiple artifacts, which can be RPM repo, Linux container, flatbacks, OS 3, ISO, whatever. And the distribution, when we put it back together, will look similar to this picture. So, I have baseline time, the minimal system, Peter will be talking about, and I have the modules running on the system. In this example, I have RPM packages here and container images here. And as we can see, as I say, the module contains packages and all the dependencies, right? So, there are some challenges, how to deal with, for example, conflicting packages in these two scenarios. So, that's why we use technology similar to software collections to avoid that, or we can use containers. And what should I choose? RPM-soil containers. Well, I don't really care because the workflow can be pretty similar. So, for example, this is pretty bad. I will just make the window... Yeah. So, in Fedora, we use DNF to install software so I can type DNF installed HTTPD, which is a web server. I can configure it by editing the configuration file. I can add my index HTML. And I can use system control style HTTPD. So, that's the workflow in current Fedora. This can be the workflow in the module Fedora. And I can't say if it's container or RPM packages. It will be pretty same for both. And maybe even few days later, I can DNF update. So, the workflow will stay the same even with modules, which is, I guess, important. So, how all of this works? So, we focus on the package instead of on the distro version. So, for example, in Fedora, we store all the packages in this git, which is distribution git. And each package has several branches. So, for example, F24, F25, which is like Fedora 24, Fedora 25, Fedora 26. And we want to change that to reflect the package. So, for example, I can have package and have branches according to the versions. So, instead of building distribution versions from the disk, so, taking like all the branches of F24 to build Fedora 25 and taking all the F25 branches to build Fedora 25, I can use modules to build modules. Yeah. So, I can, in the module MD, I can specify packages I want to include in a module and I can just mix and match whatever I want. And then I can install it on base runtime. We call it module streams. And that's like variant of the module. We don't call it versions because in some cases it would be true. But if I have, for example, a LAMP stack, I can't say what is version of a LAMP stack because I have version of database, I have version of PHP. So, that's why we call it streams. These are basically just variants of the modules. And before you install, you will be able to choose the right version or you can just download ISO with the versions preselected so the user experience will be kind of same like with Fedora today. And you can use DNF. DNF update will work. It will keep your system up to date. So, that's basically it about the module distro and now Petr will tell you something about the base runtime, which is the minimal system. If you give me your microphone. I can give you your microphone. There you go. Okay. How do I get this? I guess I will just hold it. Okay. So, as I mentioned, the base runtime is a module that sits below all the other modules, all the other stacks that we have. This part is meant generational core though. So, I will provide some background on what generational core is, what base runtime really is and the difference between them. Base runtime was originally meant to be only the lower layer of this diagram. It was meant to provide hardware abstraction layer with system tools, utilities and short libraries to be provided by other modules included in a stack that we wanted to call the generational core. This initiative is temporarily on hold, however, because of the complexity and the introduction of modularity to Fedora. We don't have all the resources and the plans and the clear idea how to actually separate it, what components go into which module and so on. Also, because the name is quite long and nobody could pronounce it, nobody really understood what generational core meant in this context. We decided to just broaden the definition of base runtime itself, include some of the system tools and libraries, not the shared libraries layer, I will get to that. But basically, when we talk about the base in February 26, we mean base runtime, not generational core. We must, we may still revisit the idea of the generational core in the future. So, the implementation. Don't worry about the picture, it's not really important if you can read it. Yeah, it's a module like any other. It's defined in a module in the file. It lists all the components that we want to build as part of the base runtime module. It provides a stable and minimal bootable system on bare metal machines, on bare virtual machines. It also defines the container base image for both system, system the N spawn and Docker files, just Docker. The components that we include were inspired by the LSB Core Set, POSIX user land and the Atomic host. Currently, we include 700 binary RPMs defined by roughly 170 source RPMs. When installed as a container base image for Docker, it's 82 packages and takes roughly 88 megs of this space. How we actually implement it is that we have this, the module I just described, which is something we want to ship, but we also have to build it somehow. So, we have another module called Temporarily Base Runtime Built Environment, which includes all the build dependencies of the components in base runtime. It also includes all the build dependencies of those build dependencies and all the entire recursive build dependency chain. Currently, it's roughly 2800 packages, which one build is roughly 6,000 binary RPMs. Although we will ship the base runtime built environment, we won't be supporting it in any way. And in the future, we hope that parts of the built environment will be split into the applications themselves and base runtime will build up and only on itself and the applications that provide the components needed to build it. So, okay, yeah, yeah, I will try. Yeah, I know, I'm just quite speaker. So, the challenges that we face when developing base runtime is that there's more than one way to do it, so, choosing the right content is the biggest challenge. We decided to go with the, yeah, with the LSB Core Set and the Atomic Host Set and the POSIX user and set I mentioned in previously, but it's definitely not the finalist. We may exclude some of those components or add something else to it. Keeping it as small as possible is important because we minimize both the memory, the space footprint, as well as the attack surface. And of course, smaller set means that we can rebuild it much more faster and we can also test it faster. Another problem is building the whole set. We started with Federal25 beta and just hope that to select those 3,000 packages we needed for the base runtime and base runtime built environment and hope that they would magically rebuild themselves and it would just work. That didn't work as expected. We got like 400 fail to build from source issues in the first run. That was mostly because Perl was removed just before Federal25 beta was branched. Many packages build require Perl but don't explicitly state the build dependency so they were failing and it was mostly caused by Autotools and Autoconf. So we fixed some of the issues and then re-based our sets to use Federal25 for this candidate 3 which and the build dependency failures, the build failures dropped to 180 back then. So we created a new tracker back then we were reporting all the FTBS issues to the package maintainers. It was mostly the missing build dependencies but it was also more poor packaging practices for example and the lack of CI in Federal25 wasn't helping. Another issue was the unresponsive maintainers. For example, when we reported those FTBS issues we gave all the maintainers two weeks to respond otherwise we would just fix it for them. In most cases we had to wait the whole two weeks because there was no response whatsoever. In other cases the package maintainer had a different opinion on how to fix the issue which is not always a problem but it just slows down the progress. Of course the discussion helps if it has potential. Another issue is that we are developing base runtime and all the modules in parallel with the traditional release. Since we are trying to build it we haven't actually got past that yet. We are working with the frozen package set and Federal is still introducing new changes in raw height that includes GCC7 package conf basically anything. Also when a maintainer fixes an issue for example when we're building the package they often rebase the package which means new dependencies and new build failures caused by the rebase. That's not always helpful. This is a picture with the build dependency graph of the current base runtime self hosting prototype. It includes those 3,000 packages and the build dependencies between them. As you can see when trying to minimize the build dependency chain both for runtime or the self hosting prototype because it just takes forever to rebuild and with all those issues it's just a nightmare. Many of those dependencies are obsolete and could be removed because they were added like years ago and nobody really cares anymore. But finding them that's the problem. I can provide SVG picture later. So what we plan to deliver in Federal26, the proof of concept based runtime module with the first version of the API hopefully all the packages in the API will also ship with the devil sub packages so people can actually use our API. That's not as easy as it sounds because the devil sub packages often will require stuff like Perl and we don't want Perl in the base runtime of course. We will also ship the build environment module although unsupported. We will ship system and container management modules such as DNF which is not part of base runtime because it moves at a different pace. The same for Docker for example. We will also ship like a proof of concept of the selection of the dynamic languages modules mostly for Python. Again the same situation as with Perl. We don't want Python in the base runtime but many of our tools including Carpium build unfortunately required. In the long term we would either ship like a small set of Python standard library including like small, small Python interpreters similar to system Python we have now but completely disconnected from the Python 3 federal ships today. We will also ship project Boltzrom which is a funny name for Federal26 server composed entirely out of modules. Unfortunately Federal26 won't have any updates. The reason is that body cannot handle any artifacts besides RPMs. We are working on fixing that though. And it will be made entirely by the majority working group members. The reason is again that we have no infrastructure for end processes to process like this for example module submission requests and changes in the package database. For Federal27 we hope that base runtime will be much more stable. All the API will be usable. The packages that are not meant to be part of API will be repackaged and shipped differently so that couldn't be accessed by anybody who is not meant to be using them. The same for the dynamic languages run times. We hope to finish the system Python and Python 3 split for instance. We will support automated builds and rebuilds so whenever you push the disk it you will both build a component and the whole module and all the modules that depend on it. We will ship updates. The update for everybody is in plan. The modularity infrastructure will be open to public so anybody will be able to submit a module and build it. There will be a new release of Federal server and more content. It doesn't have a name then. And beyond Federal cloud, Federal workstation hopefully that will include modules like NOM for example. Yeah, we hope that module releases will become our primary deliverable and the traditional release will be still available but will be composed out of modules so we will preselect several modules for the users and flatten them out into a traditional release. And it will be awesome. Yeah, so that's the end. We have plenty of time for questions. If there are any. So a question about the modules. I think more does also encapsulate things like end user and how do you ensure that the specific version of open as well doesn't conflict with other packages. There may be conflict. Right. So yeah, the question was if I, for example, have an application that depends on a library. So how do I make it not conflict with other libraries, right? So as I said, I can maybe switch back to my slides. So you can use container. So the module can be installed as a container and the container will be, we have solved that already. Or if you want to install it as an RPM, we might need to repackage the libraries of the dependencies in a way that will be installed in a separate path. Yes, yeah. No, not in containers. Containers are meant to contain everything in themselves. So just to further cover that, right? So based runtime will have some libraries that are shared amongst other things. Yeah. Well, at the beginning, it's probably going to be more stuff than we would like. So also, we would like that shared set to be smaller so that you can have independent versions. But at the beginning, it'll be like two-period. Yeah. Because I can imagine that if the entire system is built on a module, it could be quite expensive in terms of disk space and also maybe RAM consumption. So I think that being scramble-cash is the same as in RAM, then we're going to be done. Right, yeah. Yeah, there needs to be optimizations. But yeah, that's, I guess, more for the future in the RPM world. Yeah, if we have one library of a certain version, I think it would be just once on the system still. But yeah, in the containers, that's no choice, basically. But we can still use layering. So for example, I can have the base on time layer, then I can have some libraries layer, et cetera. And if they're the same for the same container, they will share both storage and runtime resources. Right. If I understand correctly, the problem you're trying to solve is the problem of installing mutually incompatible versions of the same package concurrently. That's not the only thing. Another problem is that we want, well, not a problem. We want to offer applications, not packages. And we want them to offer them an upstream ribbon life cycles, not the distribution ribbon life cycles. So we will ship the RPM repository or container depending as upstream releases new versions, not as we just decided. And what about different versions, well, different instances of packages which are normally the same version but have been built against different dependencies for whatever reason? The question is about basically the same package and basically the same version but built against different versions of dependencies. We will build all the variations against all the dependencies in the chain. So, yeah, there will be a lot of artifacts available in the repositories on the service and mirrors. But the system management tools should choose correctly which module variant to install for you. The problem is that sometimes there is no correct and incorrect. Well, there is because we track how we build it. So we know it. Okay. So your users have to accept your opinion of which is correct. So yeah, maybe I'll try to answer differently. Each module will contain all the dependencies but also the build recipe. So it will be always built the same time. So, for example, it will reference particular version of the base runtime. It will reference all basically if there are more versions of base runtime, it will be built against both, against all of them. But in a module, I can specify all the packages by branches, for example. But when I build them, the build system will save the exact commisch ashes. So it will be reproducible at all times. And yeah, the module defines all the software it needs. Oh, you can see. This concept is still requires packages, right? You can essentially, applications are still being defined by somebody on the server side, whatever that means, exactly, right? If this isn't really meant to be exposed to end users per se, except that you may have multiple versions of something available to choose from. So that doesn't mean that you can arbitrarily create your own modules as developed as you could. But that's not the concept here. The concept here is that these things are still all defined on the server side, much like our PMSR today. But the units of measure are quite a bit bigger for the most part so that the individual things can have individual life cycles. That doesn't mean that you can't go in there and twiddle with the packages, but you're going to have a different kind of user experience. Yeah, yeah. I mean, the reason why I'm asking those questions is because I mean, there's very problem that's been a lot of interest here in the last two or three years. And most people that don't know are tending towards the functional declarative approach that's used by systems such as NICs and Vigs. But you're still adhering to the imperative approach. So obviously you don't think you know to break with the other thing. No, that's a big move. So that may be the long term right answer, but that's a gut rewrite. And no way to distribute the existing distribution. So even if we were 100% sure that the mixed answer was the right answer, we can't just turn it off. So I think we're kind of walking towards that direction. Okay, so you're going to do this a kind of interim measure? Well, maybe, I would say, it's a pre-perfection. Well, in the sense that we won't know until we try it. Yeah, right? So let's put it together. It's disruptive, but it's not trade reckoning. Let's see how it works and then iterate rather than make a big bet. All works. No questions? Yes? For the containers, do we run containers or system-based spawn? For the server side, I know for the GUI, do we run containers? But for the, for example, for the server, is it containers? Do we run them on Docker? Do we run them on system-based spawn? And the answer is we will be using OCI containers, which will be able to run on Docker, run on RunC. And for example, if you have no CI image, you can imagine, I had a pretty nice example. For example, you can imagine PDF, which could be built by different tools, but you can open it in a browser, you can open it in this way, they're on your phone. So the OCI image will run on multiple container engines, but I'm not sure which would be the default. We can do both. We can do both, yeah. Can you go to factory 2.0 slide? Yeah. But I have to find it. Oh, there it is. All right, so what's the carbon footprint of factory 2.0? Well, as you can see, it will be a lot of building because, yeah, when building a module, we need to build all the packages, but I hope there will be some optimization, some nice filters to not have the carbon footprint as big as on this picture, yeah. As for the comments, we're in container run times. The container space is still a warfare zone. So if we were declarative about any particular tech there, we would be wrong. So the objective here, I think, is that we want to do kind of the OCI model, which is the open standard, and then use different technologies to run them depending on the scenario. There's some of the reasons for the system-based container stuff is because you sometimes want to run a container before something like a Docker game comes up. So you need to, just like I said, we'll just like everything else, we have to be kind of flexible about the container run times that we want to use because we're going to need different ones for different kinds of containers. I agree. Yeah, right. So do we have any more questions? All right, so I guess we can wrap up. Thanks for coming. Thank you.
 So, hi everyone. This is Peter. My name is Adam. We are from Red Hat and we are working  on a project called Fedora modularity. By the way, we have stickers for you. So, if  you come here after the talk, you can get one or two.  Only 25 of them.  Yeah, only 25, but that's fine. So, I'm not sure who of you was on Languines Talk last  year, here on FOSDEM. He was talking about how distributions are great, but not really,  and how we might need to change them to be ready for the future.  So, let's start with why they are great, because I think that's still true. They ship package  software, which means it's pretty easy to install. They also ship dependencies and everything  else with it. So, it just works. They integrate it and test it. So, nothing breaks, really.  And some important thing. It's also patched for security vulnerabilities, because, for  example, if you are a developer and you're developing your application, which is included  in the distribution, you might not have pretty much time to patch for some security holes.  So, distributions can do it for you. And distos have a lifespan, which basically means  they're released at some point, die at some point. This is, for example, Fedora. And if  we have a look at a bit more detail, we can see that I have, for example, distribution  version one. It includes some libraries and software, which is pretty much frozen for  the whole lifetime. And then I have another distribution, another version, and the same  story there. This is fine, but there are some problems, because, for example, applications  are released at their own pace, independent of distributions. And they can also come in  multiple versions, which means conflicts, right? Distributions often include only one  version of every application, because they got dependencies, they can conflict with each  other, and it would be impossible to install. So, we need to choose a version, but how to  choose the right one? That's a tricky question, because, for example, if you are a developer,  you might prefer this one, fast and latest, because that's what developers want. But if  you're a sysadmin, for example, you're running an application, you don't want it to change  so much, right? So, you might prefer the other scenario. So, I guess the right answer, we  need all of them, we can't just choose one. But how to do it? Well, there are several  solutions to this. So, for example, we can have distribution on a fast track and another  one, which is a bit slower. So, for example, this can be Fedora, this can be CentOS, or  Ubuntu LTS. If I'm a bit more crazy, maybe I can have continuous upgrades. But this still  doesn't allow me to mix and match versions as I want. So, another solution is a software  collection. How many of you heard about software collections? All right, so that's like half.  So, software collections are kind of special packages, which entail software in a separate  space. So, I can install more versions of the software on the system. But it's pretty  hacky, it doesn't work always. So, that's a solution but not ideal. Another thing would  be Linux containers. Who of you know Linux containers? Almost everyone, that's great.  So, yeah, we know it's an isolated space, which looks like a system, and it's for single  application. And I can multiple containers on system. But sometimes, if I put stuff in  the container, I need to look after for it. I might have problems with updates. And still,  I'm building containers out of a single distribution. So, if I install some different version of  something, this can get conflicted as well. So, that's why we are working on modularity,  which kind of combines everything together. And now I have like one minute short animation  with subtitles, so I won't be talking. So, if you could please read.  Or you can just read the subtitles. No. This is a talk, I'm not supposed to be talking  here. All right, so that's basically it. So, it will look like this. We will have something  called Base Runtime, which is a tiny system with a pretty long life cycle. And another  software, and the software will be like, how do you call it in English? Independent, sorry,  independent of the distribution, right? So, as I said, Base Runtime, this is what Peter  is going to talk about, which is a small system containing software with proven API stability,  like kernel and glibc, and some basic tools to make it boot. And the software run as a  module, and module is similar concept to Android applications or iPhone applications. So,  it's pretty independent of the system, and it includes all the dependencies in itself.  And as I said, modules are like applications, so it can be database, it can be Firefox,  but it can be also LAMP stack or basically any group of packages. And it will look like  this. So, I will have packages and some metadata, and I can also have packages as an API and  as dependencies, which the difference is that if I, for example, have a Firefox module,  the Firefox package will be API. This is the only package I care for, but I can have all  of the dependencies, libraries, et cetera, which are not guaranteed to be stable. But  users won't care about this, users care only about the API. And modules are defined as  a bi-module-md file, and it basically defines everything, which I'll show you right now.  So, this is the module-md. It's kind of cut off on the top, but there's nothing important.  Important is that we have some names to inversion, which is an ID of the module. Description,  licensing, some references, where the project leaves, backtracking, some optional metadata,  and here is the important part. So, I have dependencies, there is the base runtime, and  to build it, there are build-requires, which is base runtime and base runtime build. I guess  Peter will tell you more about it. And these are lists of the components. So, I can include  packages in my module, and I can include another modules in my module, which might be useful.  For example, if I have an application requiring Python, I don't have to list all the Python  packages, I just list the Python module. And in the RBM world, if I have a source package,  I can build multiple binary packages of it, but... Oh, sorry. So, in the RBM world, I  have source RBM packages, which are packages containing the source code, and if I build  it, I can build multiple binaries. And I don't want to probably include all the binaries.  There can be devl packages and stuff, so I can filter them. Thanks to this filter, here  I can specify the RPMs API, and in the future, there will be more types of API, not just  RPMs, and also something called install profile, which is interesting, basically explains how  to install the module, and I can have more ways to installing. So, for example, if I  have web server, I can install it as a production or as a developer with a bit different configuration.  So, that was module MD, and module MD is... Modules are built in something called factory  2.0. I don't have time to talk about factory 2.0, but if you can Google, for example, or  find talk recordings from DevCon, which was like last week, you can find some interesting  stuff about that. And when we build them, modules are delivered as multiple artifacts,  which can be RPM repo, Linux container, flatbacks, OS 3, ISO, whatever. And the distribution,  when we put it back together, will look similar to this picture. So, I have baseline time,  the minimal system, Peter will be talking about, and I have the modules running on the system.  In this example, I have RPM packages here and container images here. And as we can see,  as I say, the module contains packages and all the dependencies, right? So, there are  some challenges, how to deal with, for example, conflicting packages in these two scenarios.  So, that's why we use technology similar to software collections to avoid that, or we  can use containers. And what should I choose? RPM-soil containers. Well, I don't really  care because the workflow can be pretty similar. So, for example, this is pretty bad. I will  just make the window... Yeah. So, in Fedora, we use DNF to install software so I can type  DNF installed HTTPD, which is a web server. I can configure it by editing the configuration  file. I can add my index HTML. And I can use system control style HTTPD. So, that's the  workflow in current Fedora. This can be the workflow in the module Fedora. And I can't  say if it's container or RPM packages. It will be pretty same for both. And maybe even  few days later, I can DNF update. So, the workflow will stay the same even with modules, which  is, I guess, important. So, how all of this works? So, we focus on the package instead  of on the distro version. So, for example, in Fedora, we store all the packages in this  git, which is distribution git. And each package has several branches. So, for example, F24,  F25, which is like Fedora 24, Fedora 25, Fedora 26. And we want to change that to reflect  the package. So, for example, I can have package and have branches according to the versions.  So, instead of building distribution versions from the disk, so, taking like all the branches  of F24 to build Fedora 25 and taking all the F25 branches to build Fedora 25, I can use  modules to build modules. Yeah. So, I can, in the module MD, I can specify packages I  want to include in a module and I can just mix and match whatever I want. And then I  can install it on base runtime. We call it module streams. And that's like variant of  the module. We don't call it versions because in some cases it would be true. But if I have,  for example, a LAMP stack, I can't say what is version of a LAMP stack because I have  version of database, I have version of PHP. So, that's why we call it streams. These are  basically just variants of the modules. And before you install, you will be able to choose  the right version or you can just download ISO with the versions preselected so the user  experience will be kind of same like with Fedora today. And you can use DNF. DNF update  will work. It will keep your system up to date. So, that's basically it about the module  distro and now Petr will tell you something about the base runtime, which is the minimal  system.  If you give me your microphone. I can give you your microphone. There you go. Okay. How  do I get this? I guess I will just hold it. Okay. So, as I mentioned, the base runtime  is a module that sits below all the other modules, all the other stacks that we have.  This part is meant generational core though. So, I will provide some background on what  generational core is, what base runtime really is and the difference between them. Base runtime  was originally meant to be only the lower layer of this diagram. It was meant to provide  hardware abstraction layer with system tools, utilities and short libraries to be provided  by other modules included in a stack that we wanted to call the generational core. This  initiative is temporarily on hold, however, because of the complexity and the introduction  of modularity to Fedora. We don't have all the resources and the plans and the clear  idea how to actually separate it, what components go into which module and so on. Also, because  the name is quite long and nobody could pronounce it, nobody really understood what generational  core meant in this context. We decided to just broaden the definition of base runtime  itself, include some of the system tools and libraries, not the shared libraries layer,  I will get to that. But basically, when we talk about the base in February 26, we mean  base runtime, not generational core. We must, we may still revisit the idea of the generational  core in the future. So, the implementation. Don't worry about the picture, it's not really  important if you can read it. Yeah, it's a module like any other. It's defined in a  module in the file. It lists all the components that we want to build as part of the base  runtime module. It provides a stable and minimal bootable system on bare metal machines, on  bare virtual machines. It also defines the container base image for both system, system  the N spawn and Docker files, just Docker. The components that we include were inspired  by the LSB Core Set, POSIX user land and the Atomic host. Currently, we include 700 binary  RPMs defined by roughly 170 source RPMs. When installed as a container base image for Docker,  it's 82 packages and takes roughly 88 megs of this space.  How we actually implement it is that we have this, the module I just described, which is  something we want to ship, but we also have to build it somehow. So, we have another module  called Temporarily Base Runtime Built Environment, which includes all the build dependencies of  the components in base runtime. It also includes all the build dependencies of those build  dependencies and all the entire recursive build dependency chain. Currently, it's roughly  2800 packages, which one build is roughly 6,000 binary RPMs.  Although we will ship the base runtime built environment, we won't be supporting it in  any way. And in the future, we hope that parts of the built environment will be split into  the applications themselves and base runtime will build up and only on itself and the applications  that provide the components needed to build it.  So, okay, yeah, yeah, I will try. Yeah, I know, I'm just quite speaker. So, the challenges  that we face when developing base runtime is that there's more than one way to do it,  so, choosing the right content is the biggest challenge. We decided to go with the, yeah,  with the LSB Core Set and the Atomic Host Set and the POSIX user and set I mentioned  in previously, but it's definitely not the finalist. We may exclude some of those components  or add something else to it. Keeping it as small as possible is important because we  minimize both the memory, the space footprint, as well as the attack surface. And of course,  smaller set means that we can rebuild it much more faster and we can also test it faster.  Another problem is building the whole set. We started with Federal25 beta and just hope  that to select those 3,000 packages we needed for the base runtime and base runtime built  environment and hope that they would magically rebuild themselves and it would just work.  That didn't work as expected. We got like 400 fail to build from source issues in the  first run. That was mostly because Perl was removed just before Federal25 beta was branched.  Many packages build require Perl but don't explicitly state the build dependency so they  were failing and it was mostly caused by Autotools and Autoconf. So we fixed some of the issues  and then re-based our sets to use Federal25 for this candidate 3 which and the build dependency  failures, the build failures dropped to 180 back then. So we created a new tracker back  then we were reporting all the FTBS issues to the package maintainers. It was mostly  the missing build dependencies but it was also more poor packaging practices for example  and the lack of CI in Federal25 wasn't helping. Another issue was the unresponsive maintainers.  For example, when we reported those FTBS issues we gave all the maintainers two weeks to respond  otherwise we would just fix it for them. In most cases we had to wait the whole two weeks  because there was no response whatsoever. In other cases the package maintainer had a  different opinion on how to fix the issue which is not always a problem but it just  slows down the progress. Of course the discussion helps if it has potential. Another issue is  that we are developing base runtime and all the modules in parallel with the traditional  release. Since we are trying to build it we haven't actually got past that yet. We are  working with the frozen package set and Federal is still introducing new changes in raw height  that includes GCC7 package conf basically anything. Also when a maintainer fixes an issue for  example when we're building the package they often rebase the package which means new dependencies  and new build failures caused by the rebase. That's not always helpful. This is a picture  with the build dependency graph of the current base runtime self hosting prototype. It includes  those 3,000 packages and the build dependencies between them. As you can see when trying to  minimize the build dependency chain both for runtime or the self hosting prototype because  it just takes forever to rebuild and with all those issues it's just a nightmare. Many  of those dependencies are obsolete and could be removed because they were added like years  ago and nobody really cares anymore. But finding them that's the problem. I can provide SVG  picture later. So what we plan to deliver in Federal26, the proof of concept based runtime  module with the first version of the API hopefully all the packages in the API will also ship  with the devil sub packages so people can actually use our API. That's not as easy as  it sounds because the devil sub packages often will require stuff like Perl and we don't  want Perl in the base runtime of course. We will also ship the build environment module  although unsupported. We will ship system and container management modules such as  DNF which is not part of base runtime because it moves at a different pace. The same for  Docker for example. We will also ship like a proof of concept of the selection of the  dynamic languages modules mostly for Python. Again the same situation as with Perl. We  don't want Python in the base runtime but many of our tools including Carpium build  unfortunately required. In the long term we would either ship like a small set of Python  standard library including like small, small Python interpreters similar to system Python  we have now but completely disconnected from the Python 3 federal ships today. We will  also ship project Boltzrom which is a funny name for Federal26 server composed entirely  out of modules. Unfortunately Federal26 won't have any updates. The reason is that body  cannot handle any artifacts besides RPMs. We are working on fixing that though. And  it will be made entirely by the majority working group members. The reason is again that we  have no infrastructure for end processes to process like this for example module submission  requests and changes in the package database. For Federal27 we hope that base runtime will  be much more stable. All the API will be usable. The packages that are not meant to be part  of API will be repackaged and shipped differently so that couldn't be accessed by anybody who  is not meant to be using them. The same for the dynamic languages run times. We hope to  finish the system Python and Python 3 split for instance. We will support automated builds  and rebuilds so whenever you push the disk it you will both build a component and the  whole module and all the modules that depend on it. We will ship updates. The update for  everybody is in plan. The modularity infrastructure will be open to public so anybody will be  able to submit a module and build it. There will be a new release of Federal server and  more content. It doesn't have a name then. And beyond Federal cloud, Federal workstation  hopefully that will include modules like NOM for example. Yeah, we hope that module releases  will become our primary deliverable and the traditional release will be still available  but will be composed out of modules so we will preselect several modules for the users and  flatten them out into a traditional release. And it will be awesome. Yeah, so that's the  end. We have plenty of time for questions. If there are any.  So a question about the modules. I think more does also encapsulate things like end user  and how do you ensure that the specific version of open as well doesn't conflict with other  packages. There may be conflict. Right. So yeah, the question was if I, for example,  have an application that depends on a library. So how do I make it not conflict with other  libraries, right? So as I said, I can maybe switch back to my slides. So you can use container.  So the module can be installed as a container and the container will be, we have solved  that already. Or if you want to install it as an RPM, we might need to repackage the  libraries of the dependencies in a way that will be installed in a separate path. Yes,  yeah. No, not in containers. Containers are meant to contain everything in themselves.  So just to further cover that, right? So based runtime will have some libraries that  are shared amongst other things. Yeah. Well, at the beginning, it's probably going to be  more stuff than we would like. So also, we would like that shared set to be smaller so  that you can have independent versions. But at the beginning, it'll be like two-period.  Yeah. Because I can imagine that if the entire system is built on a module, it could be quite  expensive in terms of disk space and also maybe RAM consumption. So I think that being  scramble-cash is the same as in RAM, then we're going to be done. Right, yeah. Yeah, there  needs to be optimizations. But yeah, that's, I guess, more for the future in the RPM world.  Yeah, if we have one library of a certain version, I think it would be just once on  the system still. But yeah, in the containers, that's no choice, basically. But we can still  use layering. So for example, I can have the base on time layer, then I can have some libraries  layer, et cetera. And if they're the same for the same container, they will share both  storage and runtime resources. Right. If I understand correctly, the problem you're  trying to solve is the problem of installing mutually incompatible versions of the same  package concurrently. That's not the only thing. Another problem is that we want, well,  not a problem. We want to offer applications, not packages. And we want them to offer them  an upstream ribbon life cycles, not the distribution ribbon life cycles. So we will ship the RPM  repository or container depending as upstream releases new versions, not as we just decided.  And what about different versions, well, different instances of packages which are normally  the same version but have been built against different dependencies for whatever reason?  The question is about basically the same package and basically the same version but built against  different versions of dependencies. We will build all the variations against all the dependencies  in the chain. So, yeah, there will be a lot of artifacts available in the repositories  on the service and mirrors. But the system management tools should choose correctly  which module variant to install for you. The problem is that sometimes there is no correct  and incorrect. Well, there is because we track how we build it. So we know it.  Okay. So your users have to accept your opinion of which is correct.  So yeah, maybe I'll try to answer differently. Each module will contain all the dependencies  but also the build recipe. So it will be always built the same time. So, for example, it will  reference particular version of the base runtime. It will reference all basically if there  are more versions of base runtime, it will be built against both, against all of them.  But in a module, I can specify all the packages by branches, for example. But when I build  them, the build system will save the exact commisch ashes. So it will be reproducible  at all times. And yeah, the module defines all the software it needs. Oh, you can see.  This concept is still requires packages, right? You can essentially, applications are still  being defined by somebody on the server side, whatever that means, exactly, right? If this  isn't really meant to be exposed to end users per se, except that you may have multiple  versions of something available to choose from. So that doesn't mean that you can arbitrarily  create your own modules as developed as you could. But that's not the concept here. The  concept here is that these things are still all defined on the server side, much like  our PMSR today. But the units of measure are quite a bit bigger for the most part so that  the individual things can have individual life cycles. That doesn't mean that you can't  go in there and twiddle with the packages, but you're going to have a different kind  of user experience. Yeah, yeah. I mean, the reason why I'm asking those questions is because  I mean, there's very problem that's been a lot of interest here in the last two or  three years. And most people that don't know are tending towards the functional declarative  approach that's used by systems such as NICs and Vigs. But you're still adhering to the  imperative approach. So obviously you don't think you know to break with the other thing.  No, that's a big move. So that may be the long term right answer, but that's a gut rewrite.  And no way to distribute the existing distribution. So even if we were 100% sure that the mixed  answer was the right answer, we can't just turn it off. So I think we're kind of walking  towards that direction. Okay, so you're going to do this a kind of interim measure? Well,  maybe, I would say, it's a pre-perfection. Well, in the sense that we won't know until  we try it. Yeah, right? So let's put it together. It's disruptive, but it's not trade reckoning.  Let's see how it works and then iterate rather than make a big bet. All works.  No questions? Yes?  For the containers, do we run containers or system-based spawn? For the server side, I  know for the GUI, do we run containers? But for the, for example, for the server, is it  containers? Do we run them on Docker? Do we run them on system-based spawn? And the answer  is we will be using OCI containers, which will be able to run on Docker, run on RunC.  And for example, if you have no CI image, you can imagine, I had a pretty nice example.  For example, you can imagine PDF, which could be built by different tools, but you can open  it in a browser, you can open it in this way, they're on your phone. So the OCI image will  run on multiple container engines, but I'm not sure which would be the default. We can  do both. We can do both, yeah. Can you go to factory 2.0 slide? Yeah. But I have to find it.  Oh, there it is. All right, so what's the carbon footprint of factory 2.0?  Well, as you can see, it will be a lot of building because, yeah, when building a module, we need  to build all the packages, but I hope there will be some optimization, some nice filters  to not have the carbon footprint as big as on this picture, yeah.  As for the comments, we're in container run times. The container space is still a warfare  zone. So if we were declarative about any particular tech there, we would be wrong.  So the objective here, I think, is that we want to do kind of the OCI model, which is the open  standard, and then use different technologies to run them depending on the scenario.  There's some of the reasons for the system-based container stuff is because you sometimes want  to run a container before something like a Docker game comes up. So you need to, just like I  said, we'll just like everything else, we have to be kind of flexible about the container  run times that we want to use because we're going to need different ones for different  kinds of containers.  I agree.  Yeah, right.  So do we have any more questions?  All right, so I guess we can wrap up.  Thanks for coming.  Thank you.
Modularity_Generational_Core_The_future_of_Fedora_0_131.png Modularity_Generational_Core_The_future_of_Fedora_132_515.png Modularity_Generational_Core_The_future_of_Fedora_516_1907.png Modularity_Generational_Core_The_future_of_Fedora_1908_2123.png Modularity_Generational_Core_The_future_of_Fedora_2124_2615.png Modularity_Generational_Core_The_future_of_Fedora_2616_2663.png Modularity_Generational_Core_The_future_of_Fedora_2664_2891.png Modularity_Generational_Core_The_future_of_Fedora_2892_3959.png Modularity_Generational_Core_The_future_of_Fedora_3960_4223.png Modularity_Generational_Core_The_future_of_Fedora_4224_4859.png Modularity_Generational_Core_The_future_of_Fedora_4860_6899.png Modularity_Generational_Core_The_future_of_Fedora_6900_7751.png Modularity_Generational_Core_The_future_of_Fedora_7752_8663.png Modularity_Generational_Core_The_future_of_Fedora_8664_9311.png Modularity_Generational_Core_The_future_of_Fedora_9312_10319.png Modularity_Generational_Core_The_future_of_Fedora_10320_10679.png Modularity_Generational_Core_The_future_of_Fedora_10680_10787.png Modularity_Generational_Core_The_future_of_Fedora_10788_10871.png Modularity_Generational_Core_The_future_of_Fedora_10872_11303.png Modularity_Generational_Core_The_future_of_Fedora_11304_12455.png Modularity_Generational_Core_The_future_of_Fedora_12456_14195.png Modularity_Generational_Core_The_future_of_Fedora_14196_15419.png Modularity_Generational_Core_The_future_of_Fedora_15420_16007.png Modularity_Generational_Core_The_future_of_Fedora_16008_17003.png Modularity_Generational_Core_The_future_of_Fedora_17004_18911.png Modularity_Generational_Core_The_future_of_Fedora_18912_19355.png Modularity_Generational_Core_The_future_of_Fedora_19356_19967.png Modularity_Generational_Core_The_future_of_Fedora_19968_20219.png Modularity_Generational_Core_The_future_of_Fedora_20220_20279.png Modularity_Generational_Core_The_future_of_Fedora_20280_26123.png Modularity_Generational_Core_The_future_of_Fedora_26124_30983.png Modularity_Generational_Core_The_future_of_Fedora_30984_31487.png Modularity_Generational_Core_The_future_of_Fedora_31488_31547.png Modularity_Generational_Core_The_future_of_Fedora_31548_32603.png Modularity_Generational_Core_The_future_of_Fedora_32604_42755.png Modularity_Generational_Core_The_future_of_Fedora_42756_45575.png Modularity_Generational_Core_The_future_of_Fedora_45576_51371.png Modularity_Generational_Core_The_future_of_Fedora_51372_51431.png Modularity_Generational_Core_The_future_of_Fedora_51432_51479.png Modularity_Generational_Core_The_future_of_Fedora_51480_53819.png Modularity_Generational_Core_The_future_of_Fedora_53820_53939.png