<commit_before>.\" Automatically generated by Pandoc 2.2.1
.\"
.TH "debos" "1" "" "" ""
.hy
.SH NAME
.IP
.nf
\f[C]
debos\ \-\ \ Debian\ OS\ images\ builder
\f[]
.fi
.SH SYPNOSIS
.IP
.nf
\f[C]
debos\ [options]\ <recipe\ file\ in\ YAML>
debos\ [\-\-help]
\f[]
.fi
.PP
Application Options:
.IP
.nf
\f[C]
\ \ \ \ \ \ \-\-artifactdir=
\ \ \-t,\ \-\-template\-var=\ \ \ Template\ variables
\ \ \ \ \ \ \-\-debug\-shell\ \ \ \ \ Fall\ into\ interactive\ shell\ on\ error
\ \ \-s,\ \-\-shell=\ \ \ \ \ \ \ \ \ \ Redefine\ interactive\ shell\ binary\ (default:\ bash)
\ \ \ \ \ \ \-\-scratchsize=\ \ \ \ Size\ of\ disk\ backed\ scratch\ space
\ \ \-e,\ \-\-environ\-var=\ \ \ \ Environment\ variables
\f[]
.fi
.SH DESCRIPTION
.PP
debos is a tool to make the creation of various Debian\-based OS images
simpler.
While most other tools focus on specific use\-cases, debos is more meant
as a tool\-chain to make common actions trivial while providing enough
rope to do whatever tweaking that might be required behind the scene.
.PP
debos expects a YAML file as input and will run the actions listed in
the file sequentially.
These actions should be self\-contained and independent of each other.
.PP
Some of the actions provided by debos to customize and produce images
are:
.IP \[bu] 2
apt: install packages and their dependencies with `apt'
.IP \[bu] 2
debootstrap: construct the target rootfs with debootstrap
.IP \[bu] 2
download: download a single file from the internet
.IP \[bu] 2
filesystem\-deploy: deploy a root filesystem to an image previously
created
.IP \[bu] 2
image\-partition: create an image file, make partitions and format them
.IP \[bu] 2
ostree\-commit: create an OSTree commit from rootfs
.IP \[bu] 2
ostree\-deploy: deploy an OSTree branch to the image
.IP \[bu] 2
overlay: do a recursive copy of directories or files to the target
filesystem
.IP \[bu] 2
pack: create a tarball with the target filesystem
.IP \[bu] 2
raw: directly write a file to the output image at a given offset
.IP \[bu] 2
run: allows to run a command or script in the filesystem or in the host
.IP \[bu] 2
unpack: unpack files from archive in the filesystem
.PP
A full syntax description of all the debos actions can be found at:
https://godoc.org/github.com/go\-debos/debos/actions
.SH INSTALLATION (UNDER DEBIAN)
.IP
.nf
\f[C]
sudo\ apt\ install\ golang\ git\ libglib2.0\-dev\ libostree\-dev\ qemu\-system\-x86\ \\
\ \ \ \ \ qemu\-user\-static\ debootstrap\ systemd\-container
export\ GOPATH=/opt/src/gocode\ #\ or\ whatever\ suites\ your\ needs
go\ get\ \-u\ github.com/go\-debos/debos/cmd/debos
/opt/src/gocode/bin/debos\ \-\-help
\f[]
.fi
.SH SIMPLE EXAMPLE
.PP
The following example will create a arm64 image, install several
packages in it, change the file /etc/hostname to \[lq]debian\[rq] and
finally make a tarball.
.IP
.nf
\f[C]
{{\-\ $image\ :=\ or\ .image\ "debian.tgz"\ \-}}

architecture:\ arm64

actions:
\ \ \-\ action:\ debootstrap
\ \ \ \ suite:\ "buster"
\ \ \ \ components:
\ \ \ \ \ \ \-\ main
\ \ \ \ \ \ \-\ non\-free
\ \ \ \ mirror:\ https://deb.debian.org/debian
\ \ \ \ variant:\ minbase

\ \ \-\ action:\ apt
\ \ \ \ packages:\ [\ sudo,\ openssh\-server,\ adduser,\ systemd\-sysv,\ firmware\-linux\ ]

\ \ \-\ action:\ run
\ \ \ \ chroot:\ true
\ \ \ \ command:\ echo\ debian\ >\ /etc/hostname

\ \ \-\ action:\ pack
\ \ \ \ file:\ {{\ $image\ }}
\ \ \ \ compression:\ gz
\f[]
.fi
.PP
To run it, create a file named \f[C]example.yaml\f[] and run:
.IP
.nf
\f[C]
debos\ example.yaml
\f[]
.fi
.PP
The final tarball will be named \[lq]debian.tgz\[rq] if you would like
to modify this name, you can provided a different name for the variable
image like this:
.IP
.nf
\f[C]
debos\ \-t\ image:"debian\-arm64.tgz"\ example.yaml
\f[]
.fi
.SH OTHER EXAMPLES
.PP
This example builds a customized image for a Raspberry Pi 3.
https://github.com/go\-debos/debos\-recipes
.SH ENVIRONMENT VARIABLES
.PP
debos read a predefined list of environment variables from the host and
propagates it to fakemachine.
The set of environment variables is defined by environ_vars on
cmd/debos/debos.go.
Currently the list of environment variables includes the proxy
environment variables as documented at:
.PP
https://wiki.archlinux.org/index.php/proxy_settings
.PP
The list of environment variables currently exported to fakemachine is:
.IP
.nf
\f[C]
http_proxy,\ https_proxy,\ ftp_proxy,\ rsync_proxy,\ all_proxy,\ no_proxy
\f[]
.fi
.PP
While the elements of environ_vars are in lower case, for each element
both lower and upper case variants are probed on the host, and if found
propagated to fakemachine.
So if the host has the environment variables HTTP_PROXY and no_proxy
defined, both will be propagated to fakemachine respecting the case.
.PP
The command line options \[en]environ\-var and \-e can be used to
specify, overwrite, and unset environment variables for fakemachine with
the syntax:
.PP
$ debos \-e ENVIRONVAR:VALUE \&...
.PP
To unset an enviroment variable, or in other words, to prevent an
environment variable to be propagated to fakemachine, use the same
syntax without a value.
debos accept multiple \-e simultaneously.
.SH PROXY CONFIGURATION
.PP
While the proxy related environment variables are exported from the host
to fakemachine, there are two known sources of issues:
.IP \[bu] 2
Using localhost will not work from fakemachine.
Prefer using an address that is valid on your network.
debos will warn if environment variables contain localhost.
.IP \[bu] 2
In case you are running applications and/or scripts inside fakemachine
you may need to check which are the proxy environment variables they
use.
Different apps are known to use different environment variable names and
different case for environment variable names.
.SH SEE ALSO
.PP
fakemachine at https://github.com/go\-debos/fakemachine<commit_msg>doc: man: Refresh man page<commit_after>.\" Automatically generated by Pandoc 2.9.2.1
.\"
.TH "debos" "1" "" "" ""
.hy
.SH NAME
.PP
debos - Debian OS images builder
.SH SYNOPSIS
.IP
.nf
\f[C]
debos [options] <recipe file in YAML>
debos [--help]
\f[R]
.fi
.PP
Application Options:
.IP
.nf
\f[C]
  -b, --fakemachine-backend=   Fakemachine backend to use (default: auto)
      --artifactdir=           Directory for packed archives and ostree repositories (default: current directory)
  -t, --template-var=          Template variables (use -t VARIABLE:VALUE syntax)
      --debug-shell            Fall into interactive shell on error
  -s, --shell=                 Redefine interactive shell binary (default: bash) (default: /bin/bash)
      --scratchsize=           Size of disk backed scratch space
  -c, --cpus=                  Number of CPUs to use for build VM (default: 2)
  -m, --memory=                Amount of memory for build VM (default: 2048MB)
      --show-boot              Show boot/console messages from the fake machine
  -e, --environ-var=           Environment variables (use -e VARIABLE:VALUE syntax)
  -v, --verbose                Verbose output
      --print-recipe           Print final recipe
      --dry-run                Compose final recipe to build but without any real work started
      --disable-fakemachine    Do not use fakemachine.
\f[R]
.fi
.SH DESCRIPTION
.PP
debos is a tool to make the creation of various Debian-based OS images
simpler.
While most other tools focus on specific use-cases, debos is more meant
as a tool-chain to make common actions trivial while providing enough
rope to do whatever tweaking that might be required behind the scene.
.PP
debos expects a YAML file as input and will run the actions listed in
the file sequentially.
These actions should be self-contained and independent of each other.
.PP
Some of the actions provided by debos to customize and produce images
are:
.IP \[bu] 2
apt: install packages and their dependencies with `apt'
.IP \[bu] 2
debootstrap: construct the target rootfs with debootstrap
.IP \[bu] 2
download: download a single file from the internet
.IP \[bu] 2
filesystem-deploy: deploy a root filesystem to an image previously
created
.IP \[bu] 2
image-partition: create an image file, make partitions and format them
.IP \[bu] 2
ostree-commit: create an OSTree commit from rootfs
.IP \[bu] 2
ostree-deploy: deploy an OSTree branch to the image
.IP \[bu] 2
overlay: do a recursive copy of directories or files to the target
filesystem
.IP \[bu] 2
pack: create a tarball with the target filesystem
.IP \[bu] 2
raw: directly write a file to the output image at a given offset
.IP \[bu] 2
recipe: includes the recipe actions at the given path
.IP \[bu] 2
run: allows to run a command or script in the filesystem or in the host
.IP \[bu] 2
unpack: unpack files from archive in the filesystem
.PP
A full syntax description of all the debos actions can be found at:
https://godoc.org/github.com/go-debos/debos/actions
.SH INSTALLATION (DOCKER CONTAINER)
.PP
Official debos container is available:
.IP
.nf
\f[C]
docker pull godebos/debos
\f[R]
.fi
.PP
See
docker/README.md (https://github.com/go-debos/debos/blob/master/docker/README.md)
for usage.
.SH INSTALLATION (UNDER DEBIAN)
.IP
.nf
\f[C]
sudo apt install golang git libglib2.0-dev libostree-dev qemu-system-x86 \[rs]
     qemu-user-static debootstrap systemd-container
export GOPATH=/opt/src/gocode # or whatever suites your needs
go get -u github.com/go-debos/debos/cmd/debos
/opt/src/gocode/bin/debos --help
\f[R]
.fi
.SH SIMPLE EXAMPLE
.PP
The following example will create a arm64 image, install several
packages in it, change the file /etc/hostname to \[lq]debian\[rq] and
finally make a tarball.
.IP
.nf
\f[C]
{{- $image := or .image \[dq]debian.tgz\[dq] -}}

architecture: arm64

actions:
  - action: debootstrap
    suite: \[dq]buster\[dq]
    components:
      - main
      - non-free
    mirror: https://deb.debian.org/debian
    variant: minbase

  - action: apt
    packages: [ sudo, openssh-server, adduser, systemd-sysv, firmware-linux ]

  - action: run
    chroot: true
    command: echo debian > /etc/hostname

  - action: pack
    file: {{ $image }}
    compression: gz
\f[R]
.fi
.PP
To run it, create a file named \f[C]example.yaml\f[R] and run:
.IP
.nf
\f[C]
debos example.yaml
\f[R]
.fi
.PP
The final tarball will be named \[lq]debian.tgz\[rq] if you would like
to modify this name, you can provided a different name for the variable
image like this:
.IP
.nf
\f[C]
debos -t image:\[dq]debian-arm64.tgz\[dq] example.yaml
\f[R]
.fi
.SH OTHER EXAMPLES
.PP
This example builds a customized image for a Raspberry Pi 3.
https://github.com/go-debos/debos-recipes
.SH ENVIRONMENT VARIABLES
.PP
debos read a predefined list of environment variables from the host and
propagates it to fakemachine.
The set of environment variables is defined by environ_vars on
cmd/debos/debos.go.
Currently the list of environment variables includes the proxy
environment variables as documented at:
.PP
https://wiki.archlinux.org/index.php/proxy_settings
.PP
The list of environment variables currently exported to fakemachine is:
.IP
.nf
\f[C]
http_proxy, https_proxy, ftp_proxy, rsync_proxy, all_proxy, no_proxy
\f[R]
.fi
.PP
While the elements of environ_vars are in lower case, for each element
both lower and upper case variants are probed on the host, and if found
propagated to fakemachine.
So if the host has the environment variables HTTP_PROXY and no_proxy
defined, both will be propagated to fakemachine respecting the case.
.PP
The command line options \[en]environ-var and -e can be used to specify,
overwrite, and unset environment variables for fakemachine with the
syntax:
.PP
$ debos -e ENVIRONVAR:VALUE \&...
.PP
To unset an enviroment variable, or in other words, to prevent an
environment variable to be propagated to fakemachine, use the same
syntax without a value.
debos accept multiple -e simultaneously.
.SH PROXY CONFIGURATION
.PP
While the proxy related environment variables are exported from the host
to fakemachine, there are two known sources of issues:
.IP \[bu] 2
Using localhost will not work from fakemachine.
Prefer using an address that is valid on your network.
debos will warn if environment variables contain localhost.
.IP \[bu] 2
In case you are running applications and/or scripts inside fakemachine
you may need to check which are the proxy environment variables they
use.
Different apps are known to use different environment variable names and
different case for environment variable names.
.SH FAKEMACHINE BACKEND
.PP
debos (unless running debos with the \f[C]--disable-fakemachine\f[R]
argument) creates and spawns a virtual machine using
fakemachine (https://github.com/go-debos/fakemachine) and executes the
actions defined by the recipe inside the virtual machine.
This helps ensure recipes are reproducible no matter the host
environment.
.PP
Fakemachine can use different virtualisation backends to spawn the
virtualmachine, for more information see the documentation under the
fakemachine repository (https://github.com/go-debos/fakemachine).
.PP
By default the backend will automatically be selected based on what is
supported on the host machine, but this can be overridden using the
\f[C]--fakemachine-backend\f[R] option.
If no backends are supported, debos reverts to running the recipe on the
host without creating a fakemachine.
{'fraction_non_alphanumeric': 0.10038522547020168, 'fraction_numerical': 0.004909736384923332, 'mean_word_length': 3.2627173213135867, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 7, 'https://': 12, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4576307', 'n_tokens_mistral': 4352, 'n_tokens_neox': 4111, 'n_words': 1811}