<commit_before># PartyBus

Making Omnibus better, one step at a time.

## Global Upgrade Level

The upgrade level is independent of the Private Chef version number. Upgrades are either classified as a "major" upgrade or a "minor" one.

**major:** Requiring either a change in schema, a database migration, or a restart to runit or keepalived 

**minor:** Not major, e.g. single service restarts, document fixes

The current OPC upgrade version is stored in a per-host file: `/var/opt/opscode/partybus.version`

## Top-Level Upgrade DSL

### Example

```ruby
define_migration 'its_a_party' do
  maintencance_mode do

    upgrade_schema_to 42
    migrate 'migrate_to_mongodb'
    migrate 'convert_lead_to_gold'

  end
end
```

### Considerations

The reason for adding a top-level DSL around upgrades is to future proff the API. If some change is required for one of the DSL methods (e.g. upgrade_schema_to), we need the ability to support both old versions of the upgrade syntax as well as newer versions. For example:

```ruby
define_migration 'its_a_futurama_party', :api_version => 2 do

   # upgrade definition here

end
```

## Schema Upgrades

### Example

```ruby
upgrade_schema_to 10
upgrade_schema_to 11
```

## Service Restarts

### Example

```ruby
restart_service 'opscode-chef'
restart_service 'opscode-erchef'
restart_service 'opscode-webui'
restart_service 'opscode-solr'
restart_service 'couchdb'
```

### Implementation Details

#### Cluster Affecting Restarts

Restarting processes that will affect the health of the cluser (e.g. keepalived, runit) will need to be handled more carefully. The plan for these upgrades is to force the user to stop keepalived on the secondary HA node before proceeding.

## Data Migrations

### Example

```ruby
# maintenance mode will most likely be required

migrate 'some_migration_yo.script'
migrate 'some_other_migration.script'
```

## Maintenance Mode (503)

### Example

```ruby
maintenance_mode do

  # schema upgrade

  # data migration

  # service restarts

end
```

### Implementation Details

Maintenance mode should be initiated by the master back end, and cleared by each of the front ends individually.

To initiate maintenance mode we intend to expose an API that touches a file, which nginx will read to determine whether the node should be serving API requests.

To stop maintenance mode, said file will be automatically deleted (on and by the front end) at the end of the upgrade process.

### Open Questions

How do we load balance between the front ends? VIP => nginx => front-end pools? The answer affects the order in which we run upgrades / migrations on the front ends.

## Upgrade Definition Structure

TODO: write something here<commit_msg>add upgrade file layout description<commit_after># PartyBus

Making Omnibus better, one step at a time.

## Global Upgrade Level

The upgrade level is independent of the Private Chef version number. Upgrades are either classified as a "major" upgrade or a "minor" one.

**major:** Requiring either a change in schema, a database migration, or a restart to runit or keepalived 

**minor:** Not major, e.g. single service restarts, document fixes

The current OPC upgrade version is stored in a per-host file: `/var/opt/opscode/partybus.version`

## Top-Level Upgrade DSL

### Example

```ruby
define_migration 'its_a_party' do
  maintencance_mode do

    upgrade_schema_to 42
    migrate 'migrate_to_mongodb'
    migrate 'convert_lead_to_gold'

  end
end
```

### Considerations

The reason for adding a top-level DSL around upgrades is to future proff the API. If some change is required for one of the DSL methods (e.g. upgrade_schema_to), we need the ability to support both old versions of the upgrade syntax as well as newer versions. For example:

```ruby
define_migration 'its_a_futurama_party', :api_version => 2 do

   # upgrade definition here

end
```

## Schema Upgrades

### Example

```ruby
upgrade_schema_to 10
upgrade_schema_to 11
```

## Service Restarts

### Example

```ruby
restart_service 'opscode-chef'
restart_service 'opscode-erchef'
restart_service 'opscode-webui'
restart_service 'opscode-solr'
restart_service 'couchdb'
```

### Implementation Details

#### Cluster Affecting Restarts

Restarting processes that will affect the health of the cluser (e.g. keepalived, runit) will need to be handled more carefully. The plan for these upgrades is to force the user to stop keepalived on the secondary HA node before proceeding.

## Data Migrations

### Example

```ruby
# maintenance mode will most likely be required

migrate 'some_migration_yo.script'
migrate 'some_other_migration.script'
```

## Maintenance Mode (503)

### Example

```ruby
maintenance_mode do

  # schema upgrade

  # data migration

  # service restarts

end
```

### Implementation Details

Maintenance mode should be initiated by the master back end, and cleared by each of the front ends individually.

To initiate maintenance mode we intend to expose an API that touches a file, which nginx will read to determine whether the node should be serving API requests.

To stop maintenance mode, said file will be automatically deleted (on and by the front end) at the end of the upgrade process.

### Open Questions

How do we load balance between the front ends? VIP => nginx => front-end pools? The answer affects the order in which we run upgrades / migrations on the front ends.

## Upgrade Definition Structure

Upgrade definitions will be stored on the filesystem in the following fashion: #{major_ver}/#{minor_ver}_#{name}.rb. For example:

```
└── 001
    └── 001_first_upgrade.rb
    └── 002_second_upgrade.rb
└── 002
    └── 001_the_next_generation.rb
    └── 002_a_new_hope.rb
```
{'fraction_non_alphanumeric': 0.07450911640953717, 'fraction_numerical': 0.006661991584852735, 'mean_word_length': 4.517408123791102, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4898580', 'n_tokens_mistral': 1754, 'n_tokens_neox': 1633, 'n_words': 738}