<commit_before>Bifrost
=======

Bifrost is a set of Ansible playbooks that automates the task of deploying a
base image onto a set of known hardware using Ironic. It provides modular
utility for one-off operating system deployment with as few operational requirements
as reasonably possible.

This is split into roughly three steps:

- install:
  prepare the local environment by downloading and/or building machine images,
  and installing and configuring the necessary services.
- enroll:
  take as input a customizable hardware inventory file and enroll the
  listed hardware with Ironic, configuring each appropriately for deployment
  with the previously-downloaded images.
- deploy:
  instruct Ironic to deploy the operating system onto each machine.


Installation
============

The installation is split in to two parts.

The first part is a bash script which lays the basic groundwork of installing
Ansible itself.

Edit ``./playbooks/inventory/group_vars/all`` to match your environment.

- If MySQL is already installed, update mysql_password to match your local installation.
- Change network_interface to match the interface that will need to service DHCP requests.
- Change the ironic_db_password which is set by Ansible in MySQL and in Ironic's configuration file.

Then run::

  bash ./scripts/env-setup.sh
  source /opt/stack/ansible/hacking/env-setup
  cd playbooks

The second part is an Ansible playbook that installs and configures Ironic
in a stand-alone fashion.

* Keystone is NOT installed, and Ironic's API is accessible without
  authentication.  It is possible to put basic password auth on Ironic's API by
  changing the nginx configuration accordingly.
* Neutron is NOT installed. Ironic performs static IP injection via
  config-drive.
* dnsmasq is configured statically and responds to all PXE boot requests by
  chain-loading to iPXE, which then fetches the ironic-python-agent ramdisk
  from Nginx.
* standard ipmitool is used.
  TODO: make optional support for other hardware drivers

The re-execution of the playbook will cause states to be re-asserted.  If not
already present, a number of software packages including MySQL and RabbitMQ
will be installed on the host.  Python code will be re-installed regardless if
it has changed, RabbitMQ user passwords will be reset, and services will be
restarted.

Run::

  ansible-playbook -vvvv -i inventory/localhost install.yaml


Manual CLI Use
--------------

If you wish to utilize Ironic's CLI in no-auth mode, you must set two
environment variables:

- IRONIC_URL - A URL to the Ironic API, such as http://localhost:6385/
- OS_AUTH_TOKEN - Any value, such as an empty space, is required to cause the client library to send requests directly to the API.

For your ease of use, `env-vars` can be sourced to allow the CLI to connect
to a local Ironic installation operating in noauth mode.


Hardware Enrollment
===================

The following requirements are installed during the Install step above:

- openstack-infra/shade library
- openstack-infra/os-client-config

In order to enroll hardware, you will naturally need an inventory of your
hardware. When utilizing the dynamic inventory module and accompanying roles
this can be supplied in one of three ways, all of which ultimately translate
to JSON data that Ansible parses.

The original method is to utilize a CSV file.  Its format is below covered in
the `Legacy CSV File Format` section. This has a number of limitations, but
does allow a user to bulk load hardware from an inventory list with minimal
data transformations.

The newer method is to utilize a JSON or YAML document which the inventory
parser will convert and provide to Ansible.

In order to use, you will need to define the environment variable
`BIFROST_INVENTORY_SOURCE` to equal a file, which then allows you to
execute Ansible utilizing the bifrost_inventory.py file as the data source.

Conversion from CSV to JSON formats
-----------------------------------

The inventory/bifrost_inventory.py program additionally features a mode that
allows a user to convert a CSV file to the JSON data format utilizing a
`--convertcsv` command line setting when directly invoked.

Example::

  export BIFROST_INVENTORY_SOURCE=/tmp/baremetal.csv
  inventory/bifrost_inventory.py --convertcsv >/tmp/baremetal.json

JSON file format
----------------

The JSON format closely resembles the data structure that Ironic utilizes
internally.  The name, driver_info, nics, driver, and properties fields are
directly mapped through to Ironic.  This means that the data contained within
can vary from host to host, such as drivers and their parameters thus allowing
a mixed hardware environment to be defined in a single file.

Example::

  {
      "testvm1": {
        "uuid": "00000000-0000-0000-0000-000000000001",
        "driver_info": {
          "power": {
            "ssh_port": 22,
            "ssh_username": "ironic",
            "ssh_virt_type": "virsh",
            "ssh_address": "192.168.122.1",
            "ssh_key_filename": "/home/ironic/.ssh/id_rsa"
          }
        },
        "nics": [
          {
            "mac": "52:54:00:f9:32:f6"
          }
        ],
        "driver": "agent_ssh",
        "ansible_ssh_host": "192.168.122.2",
        "ipv4_address": "192.168.122.2",
        "properties": {
          "cpu_arch": "x86_64",
          "ram": "3072",
          "disk_size": "10",
          "cpus": "1"
        },
        "name": "testvm1"
      }
  }

The additional power of this format is easy configuration parameter injection,
which could potentially allow a user to provision different operating system
images onto different hardware chassis by defining the appropriate settings
in an "instance_info" variable.

Legacy CSV File Format
----------------------

The CSV file has the following columns:

0. MAC Address
1. Management username
2. Management password
3. Management Address
4. CPU Count
5. Memory size in MB
6. Disk Storage in GB
7. Flavor (Not Used)
8. Type (Not Used)
9. Host UUID
10. Host or Node name
11. Host IP Address to be set
12. ipmi_target_channel - Requires: ipmi_bridging set to single
13. ipmi_target_address - Requires: ipmi_bridging set to single
14. ipmi_transit_channel - Requires: ipmi_bridging set to dual
15. ipmi_transit_address - Requires: ipmi_bridging set to dual
16. ironic driver

Example definition::


  00:11:22:33:44:55,root,undefined,192.168.122.1,1,8192,512,NA,NA,aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee,hostname_100,192.168.2.100,,,,

This file format is fairly flexible and can be easily modified
although the enrollment and deployment playbooks utilize the model
of a host per line model in order to process through the entire
list, as well as reference the specific field items.
<commit_msg>Update Readme based on feedback from 193709<commit_after>Bifrost
=======

Bifrost is a set of Ansible playbooks that automates the task of deploying a
base image onto a set of known hardware using Ironic. It provides modular
utility for one-off operating system deployment with as few operational requirements
as reasonably possible.

This is split into roughly three steps:

- install:
  prepare the local environment by downloading and/or building machine images,
  and installing and configuring the necessary services.
- enroll:
  take as input a customizable hardware inventory file and enroll the
  listed hardware with Ironic, configuring each appropriately for deployment
  with the previously-downloaded images.
- deploy:
  instruct Ironic to deploy the operating system onto each machine.


Installation
============

The installation is split in to two parts.

The first part is a bash script which lays the basic groundwork of installing
Ansible itself.

Edit ``./playbooks/inventory/group_vars/all`` to match your environment.

- If MySQL is already installed, update mysql_password to match your local installation.
- Change network_interface to match the interface that will need to service DHCP requests.
- Change the ironic_db_password which is set by Ansible in MySQL and in Ironic's configuration file.

Then run::

  bash ./scripts/env-setup.sh
  source /opt/stack/ansible/hacking/env-setup
  cd playbooks

The second part is an Ansible playbook that installs and configures Ironic
in a stand-alone fashion.

* Keystone is NOT installed, and Ironic's API is accessible without
  authentication.  It is possible to put basic password auth on Ironic's API by
  changing the nginx configuration accordingly.
* Neutron is NOT installed. Ironic performs static IP injection via
  config-drive.
* dnsmasq is configured statically and responds to all PXE boot requests by
  chain-loading to iPXE, which then fetches the ironic-python-agent ramdisk
  from Nginx.
* standard ipmitool is used.
  TODO: make optional support for other hardware drivers

The re-execution of the playbook will cause states to be re-asserted.  If not
already present, a number of software packages including MySQL and RabbitMQ
will be installed on the host.  Python code will be re-installed regardless if
it has changed, RabbitMQ user passwords will be reset, and services will be
restarted.

Run::

  ansible-playbook -vvvv -i inventory/localhost install.yaml


Manual CLI Use
--------------

If you wish to utilize Ironic's CLI in no-auth mode, you must set two
environment variables:

- IRONIC_URL - A URL to the Ironic API, such as http://localhost:6385/
- OS_AUTH_TOKEN - Any value, such as an empty space, is required to cause the client library to send requests directly to the API.

For your ease of use, `env-vars` can be sourced to allow the CLI to connect
to a local Ironic installation operating in noauth mode.


Hardware Enrollment
===================

The following requirements are installed during the Install step above:

- openstack-infra/shade library
- openstack-infra/os-client-config

In order to enroll hardware, you will naturally need an inventory of your
hardware. When utilizing the dynamic inventory module and accompanying roles
this can be supplied in one of three ways, all of which ultimately translate
to JSON data that Ansible parses.

The original method is to utilize a CSV file. This format is covered below in
the `Legacy CSV File Format` section. This has a number of limitations, but
does allow a user to bulk load hardware from an inventory list with minimal
data transformations.

The newer method is to utilize a JSON or YAML document which the inventory
parser will convert and provide to Ansible.

In order to use, you will need to define the environment variable
`BIFROST_INVENTORY_SOURCE` to equal a file, which then allows you to
execute Ansible utilizing the bifrost_inventory.py file as the data source.

Conversion from CSV to JSON formats
-----------------------------------

The inventory/bifrost_inventory.py program additionally features a mode that
allows a user to convert a CSV file to the JSON data format utilizing a
`--convertcsv` command line setting when directly invoked.

Example::

  export BIFROST_INVENTORY_SOURCE=/tmp/baremetal.csv
  inventory/bifrost_inventory.py --convertcsv >/tmp/baremetal.json

JSON file format
----------------

The JSON format closely resembles the data structure that Ironic utilizes
internally.  The name, driver_info, nics, driver, and properties fields are
directly mapped through to Ironic.  This means that the data contained within
can vary from host to host, such as drivers and their parameters thus allowing
a mixed hardware environment to be defined in a single file.

Example::

  {
      "testvm1": {
        "uuid": "00000000-0000-0000-0000-000000000001",
        "driver_info": {
          "power": {
            "ssh_port": 22,
            "ssh_username": "ironic",
            "ssh_virt_type": "virsh",
            "ssh_address": "192.168.122.1",
            "ssh_key_filename": "/home/ironic/.ssh/id_rsa"
          }
        },
        "nics": [
          {
            "mac": "52:54:00:f9:32:f6"
          }
        ],
        "driver": "agent_ssh",
        "ansible_ssh_host": "192.168.122.2",
        "ipv4_address": "192.168.122.2",
        "properties": {
          "cpu_arch": "x86_64",
          "ram": "3072",
          "disk_size": "10",
          "cpus": "1"
        },
        "name": "testvm1"
      }
  }

The additional power of this format is easy configuration parameter injection,
which could potentially allow a user to provision different operating system
images onto different hardware chassis by defining the appropriate settings
in an "instance_info" variable.

Legacy CSV File Format
----------------------

The CSV file has the following columns:

0. MAC Address
1. Management username
2. Management password
3. Management Address
4. CPU Count
5. Memory size in MB
6. Disk Storage in GB
7. Flavor (Not Used)
8. Type (Not Used)
9. Host UUID
10. Host or Node name
11. Host IP Address to be set
12. ipmi_target_channel - Requires: ipmi_bridging set to single
13. ipmi_target_address - Requires: ipmi_bridging set to single
14. ipmi_transit_channel - Requires: ipmi_bridging set to dual
15. ipmi_transit_address - Requires: ipmi_bridging set to dual
16. ironic driver

Example definition::


  00:11:22:33:44:55,root,undefined,192.168.122.1,1,8192,512,NA,NA,aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee,hostname_100,192.168.2.100,,,,agent_ipmitool

This file format is fairly flexible and can be easily modified
although the enrollment and deployment playbooks utilize the model
of a host per line model in order to process through the entire
list, as well as reference the specific field items.

{'fraction_non_alphanumeric': 0.07380671773718327, 'fraction_numerical': 0.023865645256334708, 'mean_word_length': 3.5256666666666665, 'pattern_counts': {'":': 40, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 10, 'substrings_counts': 8, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3887785', 'n_tokens_mistral': 3946, 'n_tokens_neox': 3590, 'n_words': 1744}