Modern tooling to assist with developing applications on FreeBSD
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Chittenden, Sean
Discuss a workflow and the tooling for FreeBSD engineers to develop locally on their laptop (OS-X, Windows, or FreeBSD), and push applications to bare metal or the cloud. The tooling required to provide good automation from a developer laptop to production takes time to evolve, however this lecture will jumpstart a series of best practices for FreeBSD engineers who want to see their business applications run on FreeBSD. Developers are increasingly iterating and collaborating on golden images that are pushed to production at scale. This lecture will showcase: ) How to create a "golden image" or "stem cell image" using Vagrant, Packer, and custom FreeBSD kernels ) Perform development on a local host ) Create a golden image using Packer that a release engineering team can deploy ) How to customize the image using configuration management tools such as Ansible, Chef, Puppet, or Salt *) Deploy the image to various cloud providers such as Digital Ocean and AWS This talk will provide necessary bootstrap configuration snippets and commands required to let a developer begin using best practice development strategies. The world has evolved from administrators and developers tending to named servers and treating them as pets. Instead, we increasingly live in a world where applications and their servers are cattle and they need to be managed using abstract, automated and trusted strategies.
 So, modern tooling with FreeBSD, or performing, doing modern development on FreeBSD using modern tooling or something to that effect. So, Humayun, Sean Chitenden, I work right now at HashiCorp, been a FreeBSD user for a number of years and I'm starting to be more involved in the community again. So, I want to do a quick demo real quick to kind of explain what I mean by talking about modern tooling and explain some of the benefits of modern tooling, whatever that may be in your guys' minds, and then go through what some of the values are and some of the problems that have led to us as a community getting into the situation where we are right now, where in the cloud world we are, I will call, somewhat laggard, lagging behind where the rest of the industry is in terms of adoption. So what we're going to do in this demo here is we're going to take a packer JSON file, we're going to run it through packer, we're going to take an ISO image, run it through packer, we're going to use packer which will spit out a golden image, FreeBSD image that we can then use in order to, and then we will customize this using some background to SSH. So, please feel free to stop me along the way. So, I have a packer file here. Yes, I can. Is this better? Great. So, I have a repository GitHub that has a series of scripts here. There's one, there's a shell script in here that is going to be used to build a FreeBSD image using current and ZFS. There's a handful of arguments. None of this is too terribly fancy. And we're going to build this headless, or with headless false you can actually see what this does. Headless. Great. And a FILC on Grafite as well. You want to build training but to do this, you don't have to do this. Oh, I do need wireless offline mode for the lose. There we go. Long script. Okay. All right. So it's firing up VMware. And this is a scripted interface. I don't want to make the size different. There you go. So what this is doing in the background is it has launched a VM in a scripted way. It's booting for VSD. This establishes a VNC connection between my machine and VMware. We're waiting for this to go through. We're waiting for 45 seconds for this to finish booting. Please, it's VNC. There's no screen scraping going on here. We are doing a time delayed keyboard entry using the VMware provider. There we go. Starting to type commands. Okay. So this is useful up to a certain point. Now is it's going through a scripted install of FreeBSD. In this case, I'm using a ZFS image, which is something that we don't typically see is VMs. They're always typically UFS based on the root. So this one I created as a ZFS root. And this is interesting in the FreeBSD community because the installer means that the difference between a UFS image and a ZFS image is you push left at one point instead of pushing enter, which makes it a little problematic. You have to know exactly what you're getting into. And we will get into what that means here in a second. Configure. This is where things were breaking earlier. Please don't time out when acquiring these. Yes, it's going to break. Okay. So you can see down here that it's continuing to fire off commands because it does not acquire the lease. This is what I was trying to debug earlier. And I don't know why it's not acquiring the lease in time. All right. So I'll show you what this is doing on the behind the screens. I'm going to turn on here real quick in order to kind of semi-finish out the demo here. All right. Okay. So this is the language of Packer in the boot commands. And this is what we were seeing right here is a handful of wait commands. And either I didn't wait long enough or it's just never going to finish its lease, which is actually what the case is. So you can see here that it's firing off all of these kind of commands. Stepping through these commands right now is the only way that we can programmatically, in the course of building a new release image, create a previous D image for use in some form of a cloud environment. So at the end here, it'll eventually bounce the box and you can SSH it. Great. But that's not what's useful about Packer. What's really useful is the post processors and the provisioner, the script provisioners down here. So this allows you to, after you bring up a new VM, run arbitrary scripts, create a VM for your environment that includes all the application parameters and then upload that to Amazon Digital Ocean, Google, whoever else. You can't create an ISO today. At some point in the future, it should be very possible to programmatically create either a raw image or an ISO. It is possible to create a virtual box or a VM or image and convert them over, but that's manual and not something that's handled by the tool right now. So, go back here and see that things are still, yeah, basically hung. It worked fine on the plane. It worked fine in my office. I swear is it. I think it actually has something to do with IPv6. So anyway, so that's what this is supposed to do in that you have a previous D image that you've customized with some form of install script. You can run configuration management in that install script so that you have something that is customized, but the point is that you have a golden image coming out of Packer, that golden image you can then add to, let's say, an auto-scaling group inside of Amazon and your, create the ability to add it to an auto-scaling group so that if necessary, you can spin up additional copies in seconds as opposed to minutes after waiting for some kind of install or config management to kick off after you have like kind of a base image. So, I'm going to have a little bit of what I'm going to call an awkward chat about FreeBSD because it's a little bit of a problem child in the open source world in the sense that its adoption has really lagged behind and there's some interesting problems that have kind of led to this. Most notably, it's stable, it's debuggable, there are lots of knowledgeable administrators. It performs very well, secure, manageable, embeddable pride, right? And the last one is the interesting one, pride, actually. So, what do I mean by, and why are these problems? So, in the case of stability, right, as a community, we largely have an affinity for uptime. We like maximizing the amount of life out of a box and we frequently brag about uptimes that exceed, you know, a thousand days. And that's a problem, right? We know how to debug our stuff, okay? Thank you to Brendan Gregg. We have knowledgeable administrators and by this, the commit comments are fantastic. To this day, I still read SVN digest because the, in order for me to stay current with the community both because it's moving at a fast pace but also just to understand the motivation, there's really detailed commit messages. That's great. The man pages are fantastic. Thank you to the doc project. And we also have this fantastic reference book that everybody seems to have read. So, very knowledgeable. We, administrative staff. From a performance perspective, we are also very performant. There are large CDNs that run this and max out, you know, 100 gig interfaces. In this case, this is from WhatsApp where they are talking about stability and they were talking about how they were doing 2 million connections on a per server and what that looks like. Very, very steady performance. And at one point in time, they actually, they bragged about a million and they came back a year later and they were like, you know what? We're doing two. And in the MeetBSD presentation from two years ago, I think they were talking about 5.6 or something like that. I have to go back and watch the video again. But that's an interesting kind of thing, right? And these are problems though. We have a fixed user land kernel. Like when a box gets busy, it swaps, you kill whatever consumed all the memory and the system returns to some level of stability. Right? Again, it's, these are problems, right? We also ship it originally secure in the sense that when there is an issue, we can lock things down. So it stays, we have this steady, state stable configuration. We have lots of good utilities for doing in place upgrades, right? Merge master, Etsy update. We can create an image that is kind of perfect for our environment. It's, it's packageable. It's useful for embedded systems. And I'm going to come back to that in a second because, and one of the things that's interesting is we have this tool chain here that if implemented correctly inside of your organization is a codified way of, of describing what it is that you want, right? Make and source config give you kind of a very finely tuned system, Po-dry-a as well. Because it's embeddable, we have this, this notion that we expect the OS to be stable and long lived. And we want it to kind of work further. But because we have these assumptions, we assume that we're going to be able to front load a lot of work by adopting the technology. And that's great. But it also means that we take a lot of pride in what it is that we do. And these are, again, problems. So in, in developing this close relationship with a piece of software where we really don't want to see any kind of interruption to our system. We have a lot of interruption to our service. We have foregone an entire class of problems. And it's because these, these problems that we don't kind of generally have also mean that we've, we've oriented our focus around basically the wrong set of KPIs for users. Right? If you go back and you look at all of these, if you go back and look at all of these quote unquote problems, if you have an unstable environment and you're something that's not debuggable or you don't have performance, you're going to spin up lots of instances. You're just going to shoot the box and not ask questions later. If something's secure, you don't really care. Just leave it open. It doesn't matter because you're just going to spin up another one. You treat them like commodities. And we generally as a community don't. So as a, you know, follow on, we want to actually change the way that some of these things look, right? Stable. We care about maximizing mean time between incidents and debuggable the number of unknowns, right? We're really good at keeping that, that number low. And we're also are very interested in keeping the number of servers under management per administrator very high. On the performance side of things, we care about, you know, bits chucked per server or bytes written secure. These are all great attributes, but they're kind of problematic from a usability perspective. If we were less stable or less all of these things, we would actually be solving for a different set of problems. And as you saw earlier in my demo that actually broke because of the installer, that's kind of exactly what I mean. Like we can't create disposable infrastructure and that's what I'm getting at, right? We have largely lived in a world where server died and it's a big deal. And we want to get to a world where a new set of servers are able to automatically be spun up. And we want to be able to do that. So, yeah, so how do you do that? So what are the kind of the KPIs? Like why do we put together the infrastructure or the tooling to kind of go and optimize that? We want to go and reduce friction. So what's the effort required in order to spin up a new instance? How do we reduce that? How do we go and increase kind of our, like, quote unquote, street cred? In that we are able to have somebody go and hit random search engine of choice, type in a handful of terms and get a bunch of blog entries back. Right now I challenge you to go and look for previous decloud, whatever else, and like an exact step-by-step guide. It's not something that's easy to go and run across. So right now in order for us to go and create a new image, thank you to Microsoft Azure. What they did, they announced at this conference was a very big, useful step in that they provide this kind of stem cell image that actually lets you bypass this kind of VNC firing off random UDP packets, kind of step that we have to go through. But, you know, as a community we don't kind of think in terms of that. We think about getting an ISO and pixie-booting something. And that's an issue. So, and one of the other ones is we frequently have a development server under our desk or something like that, and that leads to differences between your development environment and in production. So, you know, in the meat space world one of the things that you would optimize for is the distance between a person and the data center, and you would do that through physical distance. We've obviously got that figured out through out of band management. But what we really want to be able to do is just, you know, very easily push something straight to the cloud and have the ability to go and spin it up with low friction. So, what's a modern workflow kind of look like? Well, you want to be able to spin up a development environment. You want to be able to test and beat the snot out of some simulated synthetic thing that's actually going to be representative of your production environment. You want to be able to go and create a golden image. Something that worked is reproducible, work and is able to be integrated with your target production environment. So, you want to have a golden image for database. You want to have a golden image for your application, a golden image for your mail server, for your bastion host, for whatever, right? What you don't want to do is have kind of like a base OS image, and then you customize and personal, you give it a personality after it's already in production there you're going to see configuration drift between my one-off in my development test environment and whatever is in production. They should be like for like with the exception of maybe an IP address, DHCP server, you know, notwithstanding. So, in order to get that, in order to get there, there's a tool that has been available for a number of years now. Some people are aware, I've actually spent a fair amount of time talking to people at this conference so far, and who here has heard of Vagrant before? More than I expected. Good. It's not, you guys are not representative of most of the people I have talked to so far. So, I'm going to go quickly through Vagrant before getting to some packer stuff, because this is an important thing, and that Vagrant allows you to, from a command line interact with a hardware virtualization layer on your laptop. So, in this case we've got a Vagrant file for FreeBSD. How many people use Vagrant for FreeBSD? Much less. Three, four people. Okay. So, let me go through this real quick. So, we've got a Vagrant file in this we've specified the API version, that's an interesting artifact of history, but we're specifying a FreeBSD guest, FreeBSD 11, that's, you know, dog food, what's current, right? Set up a synced folder, so that you share the Vagrant with Vagrant file, mount point inside of the guest with your local current directory for wherever this Vagrant file is. Use NFS, that works. And then we're going to put some provider snippets. These Vagrant files are generally pretty small, but they allow for some really nice usability, usable, you know, interactive development workflows. In the case of VMware, specify your memory size, number of CPUs, you're pretty much off to the races. For VirtualBox, there's a little bit more. I'm not actually not entirely sure. I use VMware for most of my stuff. So, VirtualBox, I've seen this on the Internet as a common config for FreeBSD. But similar, you know, you set your memory, CPU, in this case, GUI false, and see what the output is you can go and spin that up. But what that file gives you, what the Vagrant file does, is it lets you just type Vagrant up. And this will do a couple of things. It will either download the ISO and then bring up the instance, or it will, you know, if you've already got all the local artifacts, it will bring up the VM and allow you to SSHN. So, there's a handful of different providers. I'm going to come back to just a little bit here a little bit later. Because you want to be able to spin up a VMware or VirtualBox provider. So, great. Spin up your environment. Give it a key, configure it as necessary. Watch things break. This is an interesting one. FreeBSD does not work out of the box right now with current. So, okay, what's going on here? LibPam. So, we've got drift between ports and source. Do you have a handful of chuckles here? I'm wondering if this has something to do with the reason that package wasn't, or base wasn't packaged in time for 11. But it certainly was an interesting thing. So, it's like, okay, well, it turns out I'm actually running an old version of Vagrant. So, I'm going to go and pull down the current version. So, I go and do that and sit and wait about five minutes later, which is pretty good for campus network here. I pulled down the updated version from 4.30. And I tried it again, and it's still, it's helping me that it was actually already running. And I was like, well, that's interesting. So, I've already got this VM that's running, as you can see through the Vagrant status. Jump onto the box. Great. Okay, so I'm now using the updated version of Vagrant, or Vagrant. I'm using the updated version of, or the old version, I'm sorry, the reload, the box download. I'm still using the version from, that I originally had on there from March, not from April. So, I was like, okay, so, you know, in the interest of being able to randomly, you know, running around and commit violence towards VMs and just shut them down. So, destroy, try it again. All right, blew that up. It's disposable now. This is kind of like where we want to get to. So, then great, spun it up. And it blew up again. So, the box update didn't actually fix things. So, what did I have to do? Okay, so, in this case, jumped onto the box. Sudo, that doesn't work. Go and log in manually. Delete Sudo. Update, that actually did provide an update. Very happy about this. Hopefully this works. No, it doesn't. Okay, libpam, not there. Damn. All right, so, that's pretty easy. I could use libmap. Instead, I just went and created a sim link. A little bit easier and actually what I have muscle memory remember. Great, so I'm going to go and reload, which goes and logs back in. I know you probably can't read that. Logs back into the VM. Updates the configuration. And in this case, it succeeds now. And you can see way down here at the bottom, there's a little bit of thing, like Etsy, exports, whatever, mounted, NFS, shared folders. We're off to the races now. So, in my current directory, I've got a vagrant file. I can SSH into the box now. I can see that in slash vagrant, I've got, you know, synchronized file contents because NFS, great. And then I can also see that I mounted it. And that's how this happened. I did a, an NFS mount from my local, from the host, the laptop itself, into the guest at slash vagrant. And this is really useful because now I can work on my local machine, test, do whatever I want, my local editors, environments, but actually affect change or files inside of the FreeBSD environment. Go ahead. How long did you know that you were going to be in the package? There was a drift between SUDAW, which had received a shared library update and was in the ports cluster, was compiled against libpam.so.6. And in the VM that I downloaded of 11 current, it only shipped with libpam.so5. Right? So when the, the, the, the, the, yes, when the loader went to go and execute SUDAW, it, it, it failed because it couldn't resolve to find lib.so6. So, okay. So suspend then, right? So I've got this, this, this VM, which is fantastic. And I want to, you know, I can show you here my Goosefrabah and dump that off to vagrant foo logout. I can look at the file foo. It's got the right contents. And I want to shut down the, the VM. I'm actually don't shut it down. I just suspended, right? And then you can see here then that, that vagrant state, it says that it's suspended. Okay, fine. So this is great. And now I really want to kind of move to the next step where I've got this, you know, semi, I've got this development environment now that I can spin up, spin down. I can, I can test, I can have, you know, potentially dozens of vagrant files and, and VMs running either simultaneously or not. And that's really flexible. Now, what I wanted to be able to do at this point in time though is I actually want to be able to have some kind of like automated provisioning, right? You know, it's, it's nice that I can create, spin up a VM, but what I really want to do is I want to make this scriptable and I want to make this a repeatable process so that I've got either a repeatable install or image or something that is specific to the application that I'm trying to build so that it's the same on my laptop as it is in production. So in this case, right, you have this provisioner script, script, whatever it is. Okay, great. And what does that look like? Well, in this case, it's a Ruby script that's executed as a shell fo, inside of, it's a Ruby variable, executed as a shell script inside of the VM. So in this case, if this worked, I would have just done sudo package install go and run it. All right. And that's nice. That gets me a VM, but I can't take a virtual box machine and, or VM, and I can't run that in production because, you know, they're different formats. So there's something also that's kind of neat coming down the pipe that I want to just call out. And I'm really, actually really excited about this. So you can see here, we've got vagrant beehive. This is a GSOC student that is actively working on this and very active. So I'm extremely optimistic and hopeful that we're going to have a beehive provider for vagrant in the near future because then you can actually work native inside of your own environment, but using these exact same tools. So there you go. Hopefully. To be determined. But I'm optimistic. So having a vagrant file per server is nice, but what you really also want to, or what you probably want to do is be able to spin up a cluster of servers and you want to share configuration. There's one little bit here about vagrant that I wanted to highlight that I'm not sure how many people actually know about unless you're like a vagrant kind of power user, but this is really useful. This is actually how I do cluster development is I go and spin up, in this case, this is me doing Nomad development. I've got three Nomad servers here, one up to three Nomad server, %0, 2D. And then I've also got up to six Nomad clients down here. And what you do by specifying AutoStartOne and primary one up there is it allows you to type vagrant up and it will only act on the first server, in this case Nomad server 01, but I've got a config file defined for up to three servers and three clients. And this means that inside of a single vagrant file, single directory, I can go and spin up as many VMs as my laptop will reasonably support. So in this case, you can see I can just do vagrant up, vagrant suspend, great, that works as normal on just one VM, but then I can also do vagrant up on a regular expression, that's what the slash is actually interpreted as, and in this case, Nomad client, so I've got a cluster of four machines, and that's useful. So what I want to be able to do is create a dev to prod kind of workflow, because I've got the dev side of things largely squared away, but what I don't have is the ability to go and take that and push things into production. Are there questions, by the way, at this point in time? So, Packer. So what I'm really interested in doing here is filling in something that's, in the case, it's part of kind of a larger ecosystem. So we talked a bit there about Packer, or vagrant, which we use to feed kind of to Packer, and ultimately you want to be able to take that into production either through Terraform or through Nomad, which as of a couple of weeks ago does support FreeBSD in some limited, very limited capacity. So we have some Packer templates, thanks to Beardy for doing the heavy lifting and initial lifting. I was showing you some of this earlier. What this lets you do is very easily go and spin up a VM environment that is customized for your environment, and then you can just vagrant up vagrant SSH into the environment. So what does this normally look like or should look like is you should be able to, in this case, and the only reason that Shell Script exists is just as a wrapper to handle some of the options, but again, to what those options are in a sec. So what Packer does is it has the ability to download ISOs, in this case from FreeBSD, checks on them, compare it against current, runs the, spins up the VM in the background, and then you can script through things, and this is, as you can see at the bottom here, where things started to go kind of south, and I didn't actually figure out how or why. Every now and then this happens, and I didn't want to reboot my box in order to kind of debug that. But, yeah, so there's a couple of different, in this case, script files. One of them is for UFS, one of them is for ZFS. There's also for 10 and 11. And the script goes and looks at the directory to go and figure out what's current versus what's not. So I want to dump over real quick to walking through, and I'll just go through some of this stuff here. So, because there's some important components. So, like I said originally, so you pass the variables and values into Packer. So in this case, we've got a Packer, we're running the script, the program Packer build, we're passing in some variables here. These can be loaded from a file or passed in through the environment variable, or through the script, in this case we're using the script. And we're calling this template ZFS Packer file. So we're going to do a look at ZFS here. So there's some defaults here that you can see up top. We've got checksum, checksum type. These are things that I manually provided, though there's the script that we'll actually go in and do an LS and figure out what's kind of current. So there's different builders. So on my laptop I've got a VMware and VirtualBox builder support, because I have those virtualization environments installed. But there's other builders available, Azure, Google, et cetera. So these boot commands here are useful for creating that initial SSHable connection. So you can see the... So this is all of the things that you would normally step through if you were at the keyboard. What these are doing is it's sending off UDP packets to VNC to go in and perform these mechanical operations over and over and over again. In most organizations that have been dealing with 3BSD for any amount of time, they have figured out a way of building their own images and scripted away some of this with their own custom environment. But it should be easy to do this stuff out of the box. One of the things that's very frustrating about working with VNC and why we try and get out of it as fast as possible is in this case we're looking at the 11 current ZFS install. And you see these wait, 10, wait, 10, wait, 10. That's waiting 10 seconds times whatever. I think it's about 110 seconds or something like that. And you hope that it finishes that step in time. And somewhere in here there is the step where... I think it was wait, 10 here. That's the one that the DHCP bombed on. So not terribly useful to be able to interact with that at this level. But you can see like, you know, mod 7.0.0, I'm creating a user, whatever else. But really what we're trying to get down to is the permit root login and exit, wait, wait, and bomb out. So because I have VirtualBox and VMware provided in this, what this lets me do is I have the boot commands up top for VMware, ordering is not too specific. And down here I have the boot commands. They should be identical for VirtualBox. All right, now we get to the provisioners. So these scripts are executed over SSH. So you can have an array of scripts here. And this lets you customize the golden image for your BSD image on a per application basis. So this packer file is what you would use. You would have copies of this packer file, one for Postgres, let's say, if you wanted to create a database, one for your application server, one for your bastion, SSH, mail server, whatever. And in theory the only difference there is these scripts. And you can share scripts obviously between your packer images, but it allows for a repeatable process, and that's the important part that I want to get back to, keep coming back to. The provisioners step down here, these are all executed over SSH, right? All of this work up here is just to create an SSH-able image inside of FreeBSD so that you can then come down here and do the highly repeatable part of interacting with things over a character device via SSH. And then after you have completed this, there's a post-processor step that lets you do something. This is where you can potentially script and add the ability to upload your image to a cloud provider, or compress it, or do something, whatever it is that you want to do. Okay, so in this case one of the things that I like doing is I have a tendency to set log bias to throughput on my VMs because I don't really want fsync to ever cause a flush. I want it to know op, and then post install comes through and creates everything that I kind of want from my local environment. Now the Packer repo that I referenced earlier, one of the things that it lets you do is it lets you, and this is again thanks to Brad, he did this, not me, what it does is it goes and does an LS off of the ISO images, figures out which, what's the current, the latest release, and then sets up the kind of the environment variables, which is why we have this all in kind of a wrapper script. And then there's your template kind of sample vagrant file that you can then use to push up. Are there questions? I'm moved through that a lot faster than I expected, and I unfortunately have them. Yeah. There is not that I'm aware of. Yes. Okay, so we're quick, just for the benefit of folks, can you modify the ISO before you go through the installer precision? No. The follow up is why would you want to do that? So the thing that I'm interested in advocating for is that there should be an ISO image like MFSBSD that allows us to just download an image and immediately SSH into it and bypass the installer, and call into the installer in a completely scriptable way. Right, pushing left, wait one, enter, wait ten, why, enter, down, enter, enter, wait, a hundred and something seconds, hope that it's done, right, that's pretty random and less than ideal from a reproducibility perspective. So my point is you have to add that to the ISO, right, and that means that you've already invested in making this happen. In an ideal world, you should be able to go download a couple of utilities, point it at a few URLs, and then go and begin doing something without any bootstrap infrastructure. The bootstrap infrastructure required is basically zero touch and already provided. Go ahead. So that's exactly it, and I forgot to mention, Packer, you can create Docker images if you want. The whole point is to have all of the steps necessary to have a completely reproducible, verifiable, run it anywhere, kind of like build process, like release engineering in a JSON file kind of a thing. Right. So the point of Packer, yes, I understand what you're saying. Yes, so that's the reason that the config files here are specified in terms of URLs is that way potentially if there is a new ISO, let's say, because there's a security release or something like that, it will slip intentionally the version of FreeBSD that you were installing in order to stay current. What you don't want to end up in is in, and this is, I think, the problem that you were addressing or getting to earlier, which is you will end up in a world where you have a fossilized golden image that you can't reproduce. What you want to do instead is be in a codified universe where you can say what your intention is and then have the output of that be completely reproducible. Go ahead. So you can do that. In prior lives, what I've done is had to go and repack a new MFSBSD, kind of to sidestep that, and it'll just sit there with an SSH connection waiting to kind of do things after it fetches and kind of does a little setup. You can hit, you know, and step through there. You're still going through VNC though, right? It does bypass some of the trickery. I wanted to leave it kind of in the default in this case, specifically kind of make the point of navigating a UI with timing-based interactions is problematic versus something that's completely determinist, like 100% deterministic. Because really what you want to be able to do is like, if I have an error, I'm just going to try it again, right? No attachment, just like blow it away, right? It'd be nice to figure out why. Don't get me wrong, but like at the end of the day, like optimization to figure out kind of what goes on inside of the kernel is really important, and I do not want to diminish that at all. But at the same time, how do you go in like, you know, the KPI of like, what's the cost of going and deploying 100 and going from an organization that does nothing with using FBSD to an organization that goes and deploys a thousand boxes, right? Because what you, again, you know, it's all comes back to like, you know, what's that initial setup cost? You really, and a lot of times, most companies, people just don't care, right? They just need something up. So how do you bypass a lot of that? If I'm lucky here. Were there other questions, by the way? Go ahead. Instead of shell scripts? Oh, you totally could. You would use a shell script to go and like do a PKG install, answer wall, you know, go fetch your playbook, and then go do something. Yeah, absolutely. I kind of wish I would have set this as headless, because then you would be able to see it potentially stepping through things. But in theory, you shouldn't have to do that, and you should be able to just watch it work. And normally this, the entire process from like start to finish, when it's not timing out, I don't really understand why, to be honest. I have an interesting look into it, because it was working. It takes about four minutes. ZFS takes about six minutes. There would be an interesting optimization of FreeBSD Golf, is what I'm going to call it, to try and get the memory size for ZFS down to the minimum. I want to go and spin up a 128, 256 meg instance, right? Because I want to use FreeBSD. I want snapshot, but I don't want a large arc. I want like nothing, right? This is a disposable box. I deliberately know that, but I want to be able to take a snapshot, right? Or I want my date, my checksums, right? But like how do I get that down so that at the end of the day, I'm able to spin up an instance, a bunch of like 128 meg instances, right? Because that's kind of what I want. Like I can do that on UFS, but it's not the same, right? I want to go and spin up, you know, a BSD image, go and hook up and set up two EBS volumes, let's say. Have them a part of a Z pool. And then, you know, have it be a reasonably small memory footprint box, but I've got some redundancy there on the cloud service provider aspect of things. Here we go. So, yeah. Other questions? Yeah. He's aware, like the comment was, we need to talk with inside the community to figure out how to do some of this stuff. And he's right. Or like he's aware of some of this stuff. And so we're, the active conversation is going to happen there. Being able to get to, yeah. Okay. I will make sure to, I will bear, yeah, well you did the work on a lot of this. So I want to go make sure that you're a part of it. Yeah. Like one of the things that's been interesting in going from megawatt deployment data centers to cloud environments is, is there and working with large numbers of customers that work in cloud environments versus data centers is the tolerance for the setup in the bootstrap is vanishingly small inside of the cloud universe. Right. So being able to take something off of your laptop and push it straight to production, like it should take me 15, 20 minutes. I should be able to download it and it should just go. Gone are basically the days of building a custom build world. And before you can just go and provision something, right, like you can do that and lots of us do that. I do that still, right. But the attitude like the, the, there's an entire chunk of them, of the user space that's out there that has kind of evolved their, their operating parameters. So it's like, I should just be able to spin it up in minutes. And it should be like specific to my application and I need to be able to add it to an ASG auto scaling group. So we need to participate in that. This is actually going to work. All right. Well, I'll let everybody go. We don't need to watch this. If you feel free to come up and ask questions and I'm going to let this run to completion. Thank you.
 So, modern tooling with FreeBSD, or performing, doing modern development on FreeBSD using  modern tooling or something to that effect.  So, Humayun, Sean Chitenden, I work right now at HashiCorp, been a FreeBSD user for a number  of years and I'm starting to be more involved in the community again.  So, I want to do a quick demo real quick to kind of explain what I mean by talking about  modern tooling and explain some of the benefits of modern tooling, whatever that may be in  your guys' minds, and then go through what some of the values are and some of the problems  that have led to us as a community getting into the situation where we are right now,  where in the cloud world we are, I will call, somewhat laggard, lagging behind where the  rest of the industry is in terms of adoption.  So what we're going to do in this demo here is we're going to take a packer JSON file,  we're going to run it through packer, we're going to take an ISO image, run it through  packer, we're going to use packer which will spit out a golden image, FreeBSD image that  we can then use in order to, and then we will customize this using some background to SSH.  So, please feel free to stop me along the way.  So, I have a packer file here.  Yes, I can.  Is this better?  Great.  So, I have a repository GitHub that has a series of scripts here.  There's one, there's a shell script in here that is going to be used to build a FreeBSD  image using current and ZFS.  There's a handful of arguments.  None of this is too terribly fancy.  And we're going to build this headless, or with headless false you can actually see what  this does.  Headless.  Great.  And a FILC on Grafite as well.  You want to build training but to do this, you don't have to do this.  Oh, I do need wireless offline mode for the lose.  There we go.  Long script.  Okay.  All right.  So it's firing up VMware.  And this is a scripted interface.  I don't want to make the size different.  There you go.  So what this is doing in the background is it has launched a VM in a scripted way.  It's booting for VSD.  This establishes a VNC connection between my machine and VMware.  We're waiting for this to go through.  We're waiting for 45 seconds for this to finish booting.  Please, it's VNC.  There's no screen scraping going on here.  We are doing a time delayed keyboard entry using the VMware provider.  There we go.  Starting to type commands.  Okay.  So this is useful up to a certain point.  Now is it's going through a scripted install of FreeBSD.  In this case, I'm using a ZFS image, which is something that we don't typically see is VMs.  They're always typically UFS based on the root.  So this one I created as a ZFS root.  And this is interesting in the FreeBSD community because the installer means that the difference  between a UFS image and a ZFS image is you push left at one point instead of pushing enter,  which makes it a little problematic.  You have to know exactly what you're getting into.  And we will get into what that means here in a second.  Configure.  This is where things were breaking earlier.  Please don't time out when acquiring these.  Yes, it's going to break.  Okay.  So you can see down here that it's continuing to fire off commands because it does not acquire  the lease.  This is what I was trying to debug earlier.  And I don't know why it's not acquiring the lease in time.  All right.  So I'll show you what this is doing on the behind the screens.  I'm going to turn on here real quick in order to kind of semi-finish out the demo here.  All right.  Okay.  So this is the language of Packer in the boot commands.  And this is what we were seeing right here is a handful of wait commands.  And either I didn't wait long enough or it's just never going to finish its lease, which  is actually what the case is.  So you can see here that it's firing off all of these kind of commands.  Stepping through these commands right now is the only way that we can programmatically,  in the course of building a new release image, create a previous D image for use in some form  of a cloud environment.  So at the end here, it'll eventually bounce the box and you can SSH it.  Great.  But that's not what's useful about Packer.  What's really useful is the post processors and the provisioner, the script provisioners  down here.  So this allows you to, after you bring up a new VM, run arbitrary scripts, create a VM  for your environment that includes all the application parameters and then upload that  to Amazon Digital Ocean, Google, whoever else.  You can't create an ISO today.  At some point in the future, it should be very possible to programmatically create  either a raw image or an ISO.  It is possible to create a virtual box or a VM or image and convert them over, but that's  manual and not something that's handled by the tool right now.  So, go back here and see that things are still, yeah, basically hung.  It worked fine on the plane.  It worked fine in my office.  I swear is it.  I think it actually has something to do with IPv6.  So anyway, so that's what this is supposed to do in that you have a previous D image that  you've customized with some form of install script.  You can run configuration management in that install script so that you have something that  is customized, but the point is that you have a golden image coming out of Packer, that golden  image you can then add to, let's say, an auto-scaling group inside of Amazon and your, create the  ability to add it to an auto-scaling group so that if necessary, you can spin up additional  copies in seconds as opposed to minutes after waiting for some kind of install or config  management to kick off after you have like kind of a base image.  So, I'm going to have a little bit of what I'm going to call an awkward chat about FreeBSD  because it's a little bit of a problem child in the open source world in the sense that  its adoption has really lagged behind and there's some interesting problems that have  kind of led to this.  Most notably, it's stable, it's debuggable, there are lots of knowledgeable administrators.  It performs very well, secure, manageable, embeddable pride, right?  And the last one is the interesting one, pride, actually.  So, what do I mean by, and why are these problems?  So, in the case of stability, right, as a community, we largely have an affinity for uptime.  We like maximizing the amount of life out of a box and we frequently brag about uptimes  that exceed, you know, a thousand days.  And that's a problem, right?  We know how to debug our stuff, okay?  Thank you to Brendan Gregg.  We have knowledgeable administrators and by this, the commit comments are fantastic.  To this day, I still read SVN digest because the, in order for me to stay current with the  community both because it's moving at a fast pace but also just to understand the motivation,  there's really detailed commit messages.  That's great.  The man pages are fantastic.  Thank you to the doc project.  And we also have this fantastic reference book that everybody seems to have read.  So, very knowledgeable.  We, administrative staff.  From a performance perspective, we are also very performant.  There are large CDNs that run this and max out, you know, 100 gig interfaces.  In this case, this is from WhatsApp where they are talking about stability and they were talking  about how they were doing 2 million connections on a per server and what that looks like.  Very, very steady performance.  And at one point in time, they actually, they bragged about a million and they came back a year later  and they were like, you know what?  We're doing two.  And in the MeetBSD presentation from two years ago, I think they were talking about 5.6 or  something like that.  I have to go back and watch the video again.  But that's an interesting kind of thing, right?  And these are problems though.  We have a fixed user land kernel.  Like when a box gets busy, it swaps, you kill whatever consumed all the memory and the system returns to some level of stability.  Right?  Again, it's, these are problems, right?  We also ship it originally secure in the sense that when there is an issue, we can lock things down.  So it stays, we have this steady, state stable configuration.  We have lots of good utilities for doing in place upgrades, right?  Merge master, Etsy update.  We can create an image that is kind of perfect for our environment.  It's, it's packageable.  It's useful for embedded systems.  And I'm going to come back to that in a second because, and one of the things that's interesting is we have this tool chain here that if implemented  correctly inside of your organization is a codified way of, of describing what it is that you want, right?  Make and source config give you kind of a very finely tuned system, Po-dry-a as well.  Because it's embeddable, we have this, this notion that we expect the OS to be stable and long lived.  And we want it to kind of work further.  But because we have these assumptions, we assume that we're going to be able to front load a lot of work by adopting the technology.  And that's great.  But it also means that we take a lot of pride in what it is that we do.  And these are, again, problems.  So in, in developing this close relationship with a piece of software where we really don't want to see any kind of interruption to our system.  We have a lot of interruption to our service.  We have foregone an entire class of problems.  And it's because these, these problems that we don't kind of generally have also mean that we've, we've oriented our focus around basically the wrong set of KPIs for users.  Right?  If you go back and you look at all of these, if you go back and look at all of these quote unquote problems, if you have an unstable environment  and you're something that's not debuggable or you don't have performance, you're going to spin up lots of instances.  You're just going to shoot the box and not ask questions later.  If something's secure, you don't really care.  Just leave it open.  It doesn't matter because you're just going to spin up another one.  You treat them like commodities.  And we generally as a community don't.  So as a, you know, follow on, we want to actually change the way that some of these things look, right?  Stable.  We care about maximizing mean time between incidents and debuggable the number of unknowns, right?  We're really good at keeping that, that number low.  And we're also are very interested in keeping the number of servers under management per administrator very high.  On the performance side of things, we care about, you know, bits chucked per server or bytes written secure.  These are all great attributes, but they're kind of problematic from a usability perspective.  If we were less stable or less all of these things, we would actually be solving for a different set of problems.  And as you saw earlier in my demo that actually broke because of the installer, that's kind of exactly what I mean.  Like we can't create disposable infrastructure and that's what I'm getting at, right?  We have largely lived in a world where server died and it's a big deal.  And we want to get to a world where a new set of servers are able to automatically be spun up.  And we want to be able to do that.  So, yeah, so how do you do that?  So what are the kind of the KPIs?  Like why do we put together the infrastructure or the tooling to kind of go and optimize that?  We want to go and reduce friction.  So what's the effort required in order to spin up a new instance?  How do we reduce that?  How do we go and increase kind of our, like, quote unquote, street cred?  In that we are able to have somebody go and hit random search engine of choice, type in a handful of terms  and get a bunch of blog entries back.  Right now I challenge you to go and look for previous decloud, whatever else, and like an exact step-by-step guide.  It's not something that's easy to go and run across.  So right now in order for us to go and create a new image, thank you to Microsoft Azure.  What they did, they announced at this conference was a very big, useful step in that they provide this kind of stem cell image  that actually lets you bypass this kind of VNC firing off random UDP packets, kind of step that we have to go through.  But, you know, as a community we don't kind of think in terms of that.  We think about getting an ISO and pixie-booting something.  And that's an issue.  So, and one of the other ones is we frequently have a development server under our desk or something like that,  and that leads to differences between your development environment and in production.  So, you know, in the meat space world one of the things that you would optimize for is the distance between a person and the data center,  and you would do that through physical distance.  We've obviously got that figured out through out of band management.  But what we really want to be able to do is just, you know, very easily push something straight to the cloud  and have the ability to go and spin it up with low friction.  So, what's a modern workflow kind of look like?  Well, you want to be able to spin up a development environment.  You want to be able to test and beat the snot out of some simulated synthetic thing that's actually going to be representative of your production environment.  You want to be able to go and create a golden image.  Something that worked is reproducible, work and is able to be integrated with your target production environment.  So, you want to have a golden image for database.  You want to have a golden image for your application, a golden image for your mail server, for your bastion host, for whatever, right?  What you don't want to do is have kind of like a base OS image,  and then you customize and personal, you give it a personality after it's already in production  there you're going to see configuration drift between my one-off in my development test environment and whatever is in production.  They should be like for like with the exception of maybe an IP address, DHCP server, you know, notwithstanding.  So, in order to get that, in order to get there, there's a tool that has been available for a number of years now.  Some people are aware, I've actually spent a fair amount of time talking to people at this conference so far,  and who here has heard of Vagrant before?  More than I expected. Good.  It's not, you guys are not representative of most of the people I have talked to so far.  So, I'm going to go quickly through Vagrant before getting to some packer stuff,  because this is an important thing, and that Vagrant allows you to, from a command line interact with a hardware virtualization layer on your laptop.  So, in this case we've got a Vagrant file for FreeBSD.  How many people use Vagrant for FreeBSD?  Much less. Three, four people.  Okay. So, let me go through this real quick.  So, we've got a Vagrant file in this we've specified the API version, that's an interesting artifact of history,  but we're specifying a FreeBSD guest, FreeBSD 11, that's, you know, dog food, what's current, right?  Set up a synced folder, so that you share the Vagrant with Vagrant file,  mount point inside of the guest with your local current directory for wherever this Vagrant file is.  Use NFS, that works.  And then we're going to put some provider snippets.  These Vagrant files are generally pretty small, but they allow for some really nice usability,  usable, you know, interactive development workflows.  In the case of VMware, specify your memory size, number of CPUs, you're pretty much off to the races.  For VirtualBox, there's a little bit more.  I'm not actually not entirely sure.  I use VMware for most of my stuff.  So, VirtualBox, I've seen this on the Internet as a common config for FreeBSD.  But similar, you know, you set your memory, CPU, in this case, GUI false,  and see what the output is you can go and spin that up.  But what that file gives you, what the Vagrant file does, is it lets you just type Vagrant up.  And this will do a couple of things.  It will either download the ISO and then bring up the instance,  or it will, you know, if you've already got all the local artifacts,  it will bring up the VM and allow you to SSHN.  So, there's a handful of different providers.  I'm going to come back to just a little bit here a little bit later.  Because you want to be able to spin up a VMware or VirtualBox provider.  So, great.  Spin up your environment.  Give it a key, configure it as necessary.  Watch things break.  This is an interesting one.  FreeBSD does not work out of the box right now with current.  So, okay, what's going on here?  LibPam.  So, we've got drift between ports and source.  Do you have a handful of chuckles here?  I'm wondering if this has something to do with the reason that package wasn't,  or base wasn't packaged in time for 11.  But it certainly was an interesting thing.  So, it's like, okay, well, it turns out I'm actually running an old version of Vagrant.  So, I'm going to go and pull down the current version.  So, I go and do that and sit and wait about five minutes later,  which is pretty good for campus network here.  I pulled down the updated version from 4.30.  And I tried it again, and it's still, it's helping me that it was actually already running.  And I was like, well, that's interesting.  So, I've already got this VM that's running, as you can see through the Vagrant status.  Jump onto the box.  Great.  Okay, so I'm now using the updated version of Vagrant, or Vagrant.  I'm using the updated version of, or the old version, I'm sorry, the reload, the box download.  I'm still using the version from, that I originally had on there from March, not from April.  So, I was like, okay, so, you know, in the interest of being able to randomly, you know,  running around and commit violence towards VMs and just shut them down.  So, destroy, try it again.  All right, blew that up.  It's disposable now.  This is kind of like where we want to get to.  So, then great, spun it up.  And it blew up again.  So, the box update didn't actually fix things.  So, what did I have to do?  Okay, so, in this case, jumped onto the box.  Sudo, that doesn't work.  Go and log in manually.  Delete Sudo.  Update, that actually did provide an update.  Very happy about this.  Hopefully this works.  No, it doesn't.  Okay, libpam, not there.  Damn.  All right, so, that's pretty easy.  I could use libmap.  Instead, I just went and created a sim link.  A little bit easier and actually what I have muscle memory remember.  Great, so I'm going to go and reload, which goes and logs back in.  I know you probably can't read that.  Logs back into the VM.  Updates the configuration.  And in this case, it succeeds now.  And you can see way down here at the bottom, there's a little bit of thing,  like Etsy, exports, whatever, mounted, NFS, shared folders.  We're off to the races now.  So, in my current directory, I've got a vagrant file.  I can SSH into the box now.  I can see that in slash vagrant, I've got, you know,  synchronized file contents because NFS, great.  And then I can also see that I mounted it.  And that's how this happened.  I did a, an NFS mount from my local, from the host, the laptop itself,  into the guest at slash vagrant.  And this is really useful because now I can work on my local machine,  test, do whatever I want, my local editors, environments,  but actually affect change or files inside of the FreeBSD environment.  Go ahead.  How long did you know that you were going to be in the package?  There was a drift between SUDAW, which had received a shared library update  and was in the ports cluster, was compiled against libpam.so.6.  And in the VM that I downloaded of 11 current,  it only shipped with libpam.so5.  Right? So when the, the, the, the, the, yes,  when the loader went to go and execute SUDAW,  it, it, it failed because it couldn't resolve to find lib.so6.  So, okay.  So suspend then, right?  So I've got this, this, this VM, which is fantastic.  And I want to, you know, I can show you here my Goosefrabah  and dump that off to vagrant foo logout.  I can look at the file foo.  It's got the right contents.  And I want to shut down the, the VM.  I'm actually don't shut it down.  I just suspended, right?  And then you can see here then that, that vagrant state,  it says that it's suspended.  Okay, fine.  So this is great.  And now I really want to kind of move to the next step where I've got this,  you know, semi, I've got this development environment now that I can spin up,  spin down.  I can, I can test, I can have, you know, potentially dozens of vagrant files  and, and VMs running either simultaneously or not.  And that's really flexible.  Now, what I wanted to be able to do at this point in time though is I actually want to be able to have some kind of like automated provisioning, right?  You know, it's, it's nice that I can create, spin up a VM,  but what I really want to do is I want to make this scriptable and I want to make this a repeatable process  so that I've got either a repeatable install or image or something that is specific to the application that I'm trying to build  so that it's the same on my laptop as it is in production.  So in this case, right, you have this provisioner script, script, whatever it is.  Okay, great.  And what does that look like?  Well, in this case, it's a Ruby script that's executed as a shell fo, inside of, it's a Ruby variable,  executed as a shell script inside of the VM.  So in this case, if this worked, I would have just done sudo package install go and run it.  All right.  And that's nice.  That gets me a VM, but I can't take a virtual box machine and, or VM,  and I can't run that in production because, you know, they're different formats.  So there's something also that's kind of neat coming down the pipe that I want to just call out.  And I'm really, actually really excited about this.  So you can see here, we've got vagrant beehive.  This is a GSOC student that is actively working on this and very active.  So I'm extremely optimistic and hopeful that we're going to have a beehive provider for vagrant in the near future  because then you can actually work native inside of your own environment,  but using these exact same tools.  So there you go.  Hopefully.  To be determined.  But I'm optimistic.  So having a vagrant file per server is nice, but what you really also want to,  or what you probably want to do is be able to spin up a cluster of servers  and you want to share configuration.  There's one little bit here about vagrant that I wanted to highlight that I'm not sure how many people actually know about  unless you're like a vagrant kind of power user, but this is really useful.  This is actually how I do cluster development is I go and spin up,  in this case, this is me doing Nomad development.  I've got three Nomad servers here, one up to three Nomad server, %0, 2D.  And then I've also got up to six Nomad clients down here.  And what you do by specifying AutoStartOne and primary one up there is it allows you to type vagrant up  and it will only act on the first server, in this case Nomad server 01,  but I've got a config file defined for up to three servers and three clients.  And this means that inside of a single vagrant file, single directory,  I can go and spin up as many VMs as my laptop will reasonably support.  So in this case, you can see I can just do vagrant up, vagrant suspend, great,  that works as normal on just one VM, but then I can also do vagrant up on a regular expression,  that's what the slash is actually interpreted as, and in this case, Nomad client,  so I've got a cluster of four machines, and that's useful.  So what I want to be able to do is create a dev to prod kind of workflow,  because I've got the dev side of things largely squared away,  but what I don't have is the ability to go and take that and push things into production.  Are there questions, by the way, at this point in time?  So, Packer. So what I'm really interested in doing here is filling in something that's,  in the case, it's part of kind of a larger ecosystem.  So we talked a bit there about Packer, or vagrant, which we use to feed kind of to Packer,  and ultimately you want to be able to take that into production either through Terraform or through Nomad,  which as of a couple of weeks ago does support FreeBSD in some limited, very limited capacity.  So we have some Packer templates, thanks to Beardy for doing the heavy lifting and initial lifting.  I was showing you some of this earlier. What this lets you do is very easily go and spin up a VM environment  that is customized for your environment, and then you can just vagrant up vagrant SSH into the environment.  So what does this normally look like or should look like is you should be able to, in this case,  and the only reason that Shell Script exists is just as a wrapper to handle some of the options,  but again, to what those options are in a sec.  So what Packer does is it has the ability to download ISOs, in this case from FreeBSD,  checks on them, compare it against current, runs the, spins up the VM in the background,  and then you can script through things, and this is, as you can see at the bottom here,  where things started to go kind of south, and I didn't actually figure out how or why.  Every now and then this happens, and I didn't want to reboot my box in order to kind of debug that.  But, yeah, so there's a couple of different, in this case, script files.  One of them is for UFS, one of them is for ZFS. There's also for 10 and 11.  And the script goes and looks at the directory to go and figure out what's current versus what's not.  So I want to dump over real quick to walking through,  and I'll just go through some of this stuff here.  So, because there's some important components.  So, like I said originally, so you pass the variables and values into Packer.  So in this case, we've got a Packer, we're running the script, the program Packer build,  we're passing in some variables here.  These can be loaded from a file or passed in through the environment variable,  or through the script, in this case we're using the script.  And we're calling this template ZFS Packer file.  So we're going to do a look at ZFS here.  So there's some defaults here that you can see up top.  We've got checksum, checksum type.  These are things that I manually provided, though there's the script that we'll actually go in and do an LS  and figure out what's kind of current.  So there's different builders.  So on my laptop I've got a VMware and VirtualBox builder support,  because I have those virtualization environments installed.  But there's other builders available, Azure, Google, et cetera.  So these boot commands here are useful for creating that initial SSHable connection.  So you can see the...  So this is all of the things that you would normally step through if you were at the keyboard.  What these are doing is it's sending off UDP packets to VNC to go in and perform these mechanical operations over and over and over again.  In most organizations that have been dealing with 3BSD for any amount of time,  they have figured out a way of building their own images and scripted away some of this with their own custom environment.  But it should be easy to do this stuff out of the box.  One of the things that's very frustrating about working with VNC and why we try and get out of it as fast as possible  is in this case we're looking at the 11 current ZFS install.  And you see these wait, 10, wait, 10, wait, 10.  That's waiting 10 seconds times whatever.  I think it's about 110 seconds or something like that.  And you hope that it finishes that step in time.  And somewhere in here there is the step where...  I think it was wait, 10 here.  That's the one that the DHCP bombed on.  So not terribly useful to be able to interact with that at this level.  But you can see like, you know, mod 7.0.0, I'm creating a user, whatever else.  But really what we're trying to get down to is the permit root login and exit, wait, wait, and bomb out.  So because I have VirtualBox and VMware provided in this, what this lets me do is I have the boot commands  up top for VMware, ordering is not too specific.  And down here I have the boot commands.  They should be identical for VirtualBox.  All right, now we get to the provisioners.  So these scripts are executed over SSH.  So you can have an array of scripts here.  And this lets you customize the golden image for your BSD image on a per application basis.  So this packer file is what you would use.  You would have copies of this packer file, one for Postgres, let's say, if you wanted to create a database,  one for your application server, one for your bastion, SSH, mail server, whatever.  And in theory the only difference there is these scripts.  And you can share scripts obviously between your packer images,  but it allows for a repeatable process, and that's the important part that I want to get back to, keep coming back to.  The provisioners step down here, these are all executed over SSH, right?  All of this work up here is just to create an SSH-able image inside of FreeBSD  so that you can then come down here and do the highly repeatable part of interacting with things over a character device via SSH.  And then after you have completed this, there's a post-processor step that lets you do something.  This is where you can potentially script and add the ability to upload your image to a cloud provider,  or compress it, or do something, whatever it is that you want to do.  Okay, so in this case one of the things that I like doing is I have a tendency to set log bias to throughput on my VMs  because I don't really want fsync to ever cause a flush.  I want it to know op, and then post install comes through and creates everything that I kind of want from my local environment.  Now the Packer repo that I referenced earlier, one of the things that it lets you do is it lets you,  and this is again thanks to Brad, he did this, not me, what it does is it goes and does an LS off of the ISO images,  figures out which, what's the current, the latest release, and then sets up the kind of the environment variables,  which is why we have this all in kind of a wrapper script.  And then there's your template kind of sample vagrant file that you can then use to push up.  Are there questions? I'm moved through that a lot faster than I expected, and I unfortunately have them.  Yeah.  There is not that I'm aware of.  Yes.  Okay, so we're quick, just for the benefit of folks, can you modify the ISO before you go through the installer precision?  No. The follow up is why would you want to do that?  So the thing that I'm interested in advocating for is that there should be an ISO image like MFSBSD that allows us to just download an image and immediately SSH into it and bypass the installer,  and call into the installer in a completely scriptable way.  Right, pushing left, wait one, enter, wait ten, why, enter, down, enter, enter, wait, a hundred and something seconds, hope that it's done, right, that's pretty random and less than ideal from a reproducibility perspective.  So my point is you have to add that to the ISO, right, and that means that you've already invested in making this happen.  In an ideal world, you should be able to go download a couple of utilities, point it at a few URLs, and then go and begin doing something without any bootstrap infrastructure.  The bootstrap infrastructure required is basically zero touch and already provided.  Go ahead.  So that's exactly it, and I forgot to mention, Packer, you can create Docker images if you want.  The whole point is to have all of the steps necessary to have a completely reproducible, verifiable, run it anywhere, kind of like build process, like release engineering in a JSON file kind of a thing.  Right.  So the point of Packer, yes, I understand what you're saying.  Yes, so that's the reason that the config files here are specified in terms of URLs is that way potentially if there is a new ISO, let's say, because there's a security release or something like that,  it will slip intentionally the version of FreeBSD that you were installing in order to stay current.  What you don't want to end up in is in, and this is, I think, the problem that you were addressing or getting to earlier, which is you will end up in a world where you have a fossilized golden image that you can't reproduce.  What you want to do instead is be in a codified universe where you can say what your intention is and then have the output of that be completely reproducible.  Go ahead.  So you can do that.  In prior lives, what I've done is had to go and repack a new MFSBSD, kind of to sidestep that, and it'll just sit there with an SSH connection waiting to kind of do things after it fetches and kind of does a little setup.  You can hit, you know, and step through there.  You're still going through VNC though, right?  It does bypass some of the trickery. I wanted to leave it kind of in the default in this case, specifically kind of make the point of navigating a UI with timing-based interactions is problematic versus something that's completely determinist, like 100% deterministic.  Because really what you want to be able to do is like, if I have an error, I'm just going to try it again, right?  No attachment, just like blow it away, right?  It'd be nice to figure out why. Don't get me wrong, but like at the end of the day, like optimization to figure out kind of what goes on inside of the kernel is really important, and I do not want to diminish that at all.  But at the same time, how do you go in like, you know, the KPI of like, what's the cost of going and deploying 100 and going from an organization that does nothing with using FBSD to an organization that goes and deploys a thousand boxes, right?  Because what you, again, you know, it's all comes back to like, you know, what's that initial setup cost?  You really, and a lot of times, most companies, people just don't care, right?  They just need something up.  So how do you bypass a lot of that?  If I'm lucky here.  Were there other questions, by the way?  Go ahead.  Instead of shell scripts? Oh, you totally could. You would use a shell script to go and like do a PKG install, answer wall, you know, go fetch your playbook, and then go do something. Yeah, absolutely.  I kind of wish I would have set this as headless, because then you would be able to see it potentially stepping through things. But in theory, you shouldn't have to do that, and you should be able to just watch it work.  And normally this, the entire process from like start to finish, when it's not timing out, I don't really understand why, to be honest.  I have an interesting look into it, because it was working.  It takes about four minutes.  ZFS takes about six minutes.  There would be an interesting optimization of FreeBSD Golf, is what I'm going to call it, to try and get the memory size for ZFS down to the minimum.  I want to go and spin up a 128, 256 meg instance, right?  Because I want to use FreeBSD. I want snapshot, but I don't want a large arc. I want like nothing, right?  This is a disposable box. I deliberately know that, but I want to be able to take a snapshot, right?  Or I want my date, my checksums, right?  But like how do I get that down so that at the end of the day, I'm able to spin up an instance, a bunch of like 128 meg instances, right?  Because that's kind of what I want.  Like I can do that on UFS, but it's not the same, right?  I want to go and spin up, you know, a BSD image, go and hook up and set up two EBS volumes, let's say.  Have them a part of a Z pool.  And then, you know, have it be a reasonably small memory footprint box, but I've got some redundancy there on the cloud service provider aspect of things.  Here we go.  So, yeah.  Other questions?  Yeah. He's aware, like the comment was, we need to talk with inside the community to figure out how to do some of this stuff.  And he's right.  Or like he's aware of some of this stuff.  And so we're, the active conversation is going to happen there.  Being able to get to, yeah.  Okay. I will make sure to, I will bear, yeah, well you did the work on a lot of this.  So I want to go make sure that you're a part of it.  Yeah. Like one of the things that's been interesting in going from megawatt deployment data centers to cloud environments is, is there and working with large numbers of customers that work in cloud environments versus data centers is the tolerance for the setup in the bootstrap is vanishingly small inside of the cloud universe.  Right. So being able to take something off of your laptop and push it straight to production, like it should take me 15, 20 minutes.  I should be able to download it and it should just go.  Gone are basically the days of building a custom build world.  And before you can just go and provision something, right, like you can do that and lots of us do that.  I do that still, right.  But the attitude like the, the, there's an entire chunk of them, of the user space that's out there that has kind of evolved their, their operating parameters.  So it's like, I should just be able to spin it up in minutes.  And it should be like specific to my application and I need to be able to add it to an ASG auto scaling group.  So we need to participate in that.  This is actually going to work.  All right. Well, I'll let everybody go. We don't need to watch this. If you feel free to come up and ask questions and I'm going to let this run to completion.  Thank you.
modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_0_2195.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_2196_2435.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_2436_2507.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_2508_3311.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_3312_3383.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_3384_5459.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_5460_5975.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_5976_6107.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_6108_7007.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_7008_7067.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_7068_7163.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_7164_7739.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_7740_7919.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_7920_8015.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_8016_8891.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_8892_9011.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_9012_9275.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_9276_9623.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_9624_10055.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_10056_10139.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_10140_14831.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_14832_18011.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_18012_19019.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_19020_19547.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_19548_19619.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_19620_44999.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_45000_45167.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_45168_45299.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_45300_45455.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_45456_45563.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_45564_45611.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_45612_45683.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_45684_45851.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_45852_46187.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_46188_46331.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_46332_48383.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_48384_48455.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_48456_48791.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_48792_49379.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_49380_49547.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_49548_50507.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_50508_51635.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_51636_54335.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_54336_56315.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_56316_57383.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_57384_58115.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_58116_59459.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_59460_60155.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_60156_61115.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_61116_61595.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_61596_62075.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_62076_62291.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_62292_62531.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_62532_62627.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_62628_63755.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_63756_64607.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_64608_65387.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_65388_66011.png modern_tooling_to_assist_with_developing_applicati_weUYEX9WRS4_66012_67302.png