<commit_before>Openstack Nova Solver Scheduler
===============================

Solver Scheduler is an Openstack Nova Scheduler driver that provides a smarter, complex constraints optimization based resource scheduling in Nova.  It is a pluggable scheduler driver, that can leverage existing complex constraint solvers available in open source such as PULP, CVXOPT, Google OR-TOOLS, etc. It can be easily extended to add complex constraint models for various use cases, written using any of the available open source constraint solving frameworks. 

Getting Started
---------------
The project code has to be patched onto an existing installation of Openstack Nova, as it is a pluggable Nova scheduler driver.

Key modules
-----------

* The new scheduler driver module:

    nova/scheduler/solver_scheduler.py

* A patched version of host_manager module from the master Nova project, with a new method:

    nova/scheduler/host_manager.py

* The code includes a reference implementation of a solver that models the scheduling problem as a Linear Programming model, written using the PULP LP modeling language. It uses a PULP_CBC_CMD, which is a packaged constraint solver, included in the coinor-pulp python package.

    nova/scheduler/solvers/hosts_pulp_solver.py

* There are two examples of pluggable solvers using coinor-pulp or or-tools package, where costs functions and linear constraints can be plugged into the solver.

    nova/scheduler/solvers/hosts_pulp_solver_v2.py  
    nova/scheduler/solvers/hosts_ortools_linear_solver.py

Additional modules
------------------

* The cost functions pluggable to solver:

    nova/scheduler/solvers/costs/ram_cost.py  
    nova/scheduler/solvers/costs/ip_distance_cost.py  

* The linear constraints that are pluggable to solver:

    nova/scheduler/solvers/linearconstraints/active_host_constraint.py  
    nova/scheduler/solvers/linearconstraints/affinity_constraint.py  
    nova/scheduler/solvers/linearconstraints/num_hosts_per_instance_constraint.py  
    nova/scheduler/solvers/linearconstraints/max_instances_per_host_constraint.py  
    nova/scheduler/solvers/linearconstraints/resource_allocation_constraint.py  

Requirements
------------

* coinor.pulp>=1.0.4
* or-tools>=1.0.2902 (Alternative. There is a known issue with or-tools package. See below.)

Known Issues
------------

* In some cases, the installation of or-tools package may cause unexpected crash of multiple OpenStack services due to a dependency problem. There has not been report of this issue for usage in Devstack environment.

Installing Solver Scheduler
---------------------------

The Solver Scheduler Manger will allow you to manage the solver scheduler in your openstack installation.

* **Note:**  
    - This is an **alpha** version, which was tested on **Ubuntu 12.04** and **OpenStack Havana** only.
    - It is recommended that a backup of the following files be kept before using this manager:  
        /etc/nova/nova.conf  
        nova/scheduler/host_manager.py  

To install the manager, run:  
```curl https://raw.github.com/CiscoSystems/nova-solver-scheduler/master/install_manager | sudo bash```

To install solver scheduler with this manager, use the following command as root:  
```
solver-scheduler install
```
To manage the solver scheduler, use one of the following commands as root:  
```
solver-scheduler activate
solver-scheduler deactivate
solver-scheduler remove
solver-scheduler update
solver-scheduler help
```

Configurations
--------------

* This is a configuration sample for the solver-scheduler. Please add these options to nova.conf.
* Note:
    - Instead of being added, the following existing options should be updated with new values: scheduler_driver
    - The module 'nova.scheduler.solvers.hosts_pulp_solver' is self-inclusive and non-pluggable for costs and constraints. Therefore, if the option 'scheduler_host_solver' is set to use this module, there is no need for additional costs/constraints configurations.
    - Please refer to the 'Configuration Details' section below for proper configuration of costs and constraints.

```
#
# Solver Scheduler Options
#

# Default driver to use for the scheduler
scheduler_driver = nova.scheduler.solver_scheduler.ConstraintSolverScheduler

# Default solver to use for the solver scheduler
scheduler_host_solver = nova.scheduler.solvers.hosts_pulp_solver_v2.HostsPulpSolver

# Cost functions to use in the linear solver
scheduler_solver_costs = RamCost, IpDistanceCost

# Weight of each cost (every cost function used should be given a weight.)
scheduler_solver_cost_weights = RamCost:0.25, IpDistanceCost:0.75

# Constraints used in the solver
scheduler_solver_constraints = ActiveHostConstraint, NumHostsPerInstanceConstraint, MaxDiskAllocationPerHostConstraint, MaxRamAllocationPerHostConstraint

# Way of ram usage
# set negative for balancing
# set positive for stacking
ram_cost_optimization_multiplier = -1

# Virtual-to-physical disk allocation ratio
linearconstraint_disk_allocation_ratio = 1.0

# Virtual-to-physical ram allocation ratio
linearconstraint_ram_allocation_ratio = 1.0
```

Configuration Details
---------------------

* Available costs  

    - **RamCost**  
        Help to balance (or stack) ram usage of hosts.  
        The following option should be set in configuration when using this cost:  
        ```ram_cost_optimization_multiplier = <a real number>```  
        Set the multiplier to negative number for balanced ram usage,  
        set the multiplier to positive number for stacked ram usage.  
    
    - **IpDistanceCost**  
        Help to place instances close to a set of volumes.  
        The distance between instances and volumes are evaluated by using ip address.  
        The following scheuler hint is expected when using this cost:  
        ```ip_distance_cost_volume_id_list = <a list of volume ids>```  

* Available linear constraints  

    - **ActiveHostConstraint**  
        Only enabled and operational hosts are allowed in solution.  
        Normally this constraint should always be enabled.  
    
    - **AffinityConstraint**  
        Force instances to be placed at either different or same hosts as a given set of instances.  
        The following scheduler hint is expected when using this constraint:  
        ```different_host = <a list of instance uuids>``` or ```same_host= <a list of instance uuids>```  
    
    - **NumHostsPerInstanceConstraint**  
        The purpose of this constraint is to avoid trivial solution (i.e. instances placed nowhere).  
        Normally this constraint should always be enabled.  
    
    - **MaxInstancesPerHostConstraint**  
        Specify the maximum number of instances placed in each host in each scheduling process.  
        The following scheduler hint is expected when using this constraint:  
        ```max_instances_per_host = <a positive integer>```  
        By default, max_instances_per_host = 1, resulting in an anti-affinity placement solution.  
    
    - **MaxDiskAllocationPerHostConstraint**  
        Cap the virtual disk allocation of hosts.  
        The following option should be set in configuration when using this constraint:  
        ```linearconstraint_disk_allocation_ratio = <a positive real number>``` (virtual-to-physical disk allocation ratio, if >1.0 then over-allocation is allowed.)  
    
    - **MaxRamAllocationPerHostConstraint**  
        Cap the virtual ram allocation of hosts.  
        The following option should be set in configuration when using this constraint:  
        ```linearconstraint_ram_allocation_ratio = <a positive real number>``` (virtual-to-physical ram allocation ratio, if >1.0 then over-allocation is allowed.)  
    
    - **MaxVcpuAllocationPerHostConstraint**  
        Cap the vcpu allocation of hosts.  

Examples
--------

This is an example usage for creating VMs with volume affinity using the solver scheduler.

* Install the solver scheduler.

* Update the nova.conf with following options:<commit_msg>Updated README.<commit_after>Openstack Nova Solver Scheduler
===============================

Solver Scheduler is an Openstack Nova Scheduler driver that provides a smarter, complex constraints optimization based resource scheduling in Nova.  It is a pluggable scheduler driver, that can leverage existing complex constraint solvers available in open source such as PULP, CVXOPT, Google OR-TOOLS, etc. It can be easily extended to add complex constraint models for various use cases, written using any of the available open source constraint solving frameworks. 

Getting Started
---------------
The project code has to be patched onto an existing installation of Openstack Nova, as it is a pluggable Nova scheduler driver.

Key modules
-----------

* The new scheduler driver module:

    nova/scheduler/solver_scheduler.py

* A patched version of host_manager module from the master Nova project, with a new method:

    nova/scheduler/host_manager.py

* The code includes a reference implementation of a solver that models the scheduling problem as a Linear Programming model, written using the PULP LP modeling language. It uses a PULP_CBC_CMD, which is a packaged constraint solver, included in the coinor-pulp python package.

    nova/scheduler/solvers/hosts_pulp_solver.py

* The pluggable solvers using coinor-pulp package, where costs functions and linear constraints can be plugged into the solver.

    nova/scheduler/solvers/pluggable_hosts_pulp_solver.py

Additional modules
------------------

* The cost functions pluggable to solver:

    nova/scheduler/solvers/costs/ram_cost.py  
    nova/scheduler/solvers/costs/volume_affinity_cost.py  

* The linear constraints that are pluggable to solver:

    nova/scheduler/solvers/linearconstraints/active_host_constraint.py  
    nova/scheduler/solvers/linearconstraints/all_hosts_constraint.py  
    nova/scheduler/solvers/linearconstraints/availability_zone_constraint.py  
    nova/scheduler/solvers/linearconstraints/different_host_constraint.py  
    nova/scheduler/solvers/linearconstraints/same_host_constraint.py  
    nova/scheduler/solvers/linearconstraints/io_ops_constraint.py  
    nova/scheduler/solvers/linearconstraints/max_disk_allocation_constraint.py  
    nova/scheduler/solvers/linearconstraints/max_ram_allocation_constraint.py  
    nova/scheduler/solvers/linearconstraints/max_vcpu_allocation_constraint.py  
    nova/scheduler/solvers/linearconstraints/max_instances_per_host_constraint.py  
    nova/scheduler/solvers/linearconstraints/non_trivial_solution_constraint.py  

Requirements
------------

* coinor.pulp>=1.0.4

Installing Solver Scheduler
---------------------------

The Solver Scheduler Manger will allow you to manage the solver scheduler in your openstack installation.

* **Note:**  
    - This is an **alpha** version, which was tested on **Ubuntu 12.04** and **OpenStack Havana** only.
    - It is recommended that a backup of the following files be kept before using this manager:  
        /etc/nova/nova.conf  
        nova/scheduler/host_manager.py  

To install the manager, run:  
```curl https://raw.github.com/CiscoSystems/nova-solver-scheduler/master/install_manager | sudo bash```

To install solver scheduler with this manager, use the following command as root:  
```
solver-scheduler install
```
To manage the solver scheduler, use one of the following commands as root:  
```
solver-scheduler activate
solver-scheduler deactivate
solver-scheduler remove
solver-scheduler update
solver-scheduler help
```

Configurations - Getting Started
--------------------------------

* This is a configuration sample for the solver-scheduler. Please add/modify these options to nova.conf.
* Note:
    - Instead of being added, the following existing options should be updated with new values: scheduler_driver
    - The module 'nova.scheduler.solvers.hosts_pulp_solver' is self-inclusive and non-pluggable for costs and constraints. Therefore, if the option 'scheduler_host_solver' is set to use this module, there is no need for additional costs/constraints configurations.
    - Please refer to the 'Configuration Details' section below for proper configuration of costs and constraints.

```
#
# Solver Scheduler Options
#

# Default driver to use for the scheduler
scheduler_driver = nova.scheduler.solver_scheduler.ConstraintSolverScheduler

# Default solver to use for the solver scheduler
scheduler_host_solver = nova.scheduler.solvers.hosts_pulp_solver_v2.HostsPulpSolver

# Cost functions to use in the linear solver
scheduler_solver_costs = RamCost, IpDistanceCost

# Weight of each cost (every cost function used should be given a weight.)
scheduler_solver_cost_weights = RamCost:0.25, IpDistanceCost:0.75

# Constraints used in the solver
scheduler_solver_constraints = ActiveHostConstraint, NumHostsPerInstanceConstraint, MaxDiskAllocationPerHostConstraint, MaxRamAllocationPerHostConstraint

# Way of ram usage
# set negative for balancing
# set positive for stacking
ram_cost_optimization_multiplier = -1

# Virtual-to-physical disk allocation ratio
linearconstraint_disk_allocation_ratio = 1.0

# Virtual-to-physical ram allocation ratio
linearconstraint_ram_allocation_ratio = 1.0
```

Configuration Details
---------------------

* Available costs  

    - **RamCost**  
        Help to balance (or stack) ram usage of hosts.  
        The following option should be set in configuration when using this cost:  
        ```ram_cost_optimization_multiplier = <a real number>```  
        Set the multiplier to negative number for balanced ram usage,  
        set the multiplier to positive number for stacked ram usage.  
    
    - **VolumeAffinityCost**  
        Help to place instances at the same host as a specific volume, if possible.  
        In order to use this cost, you need to pass a hint to the scheduler on booting a server.
        ```nova boot ... --hint affinity_volume_id=<id of the affinity volume> ...```

* Available linear constraints  

    - **ActiveHostConstraint**  
        By enabling this constraint, only enabled and operational hosts are allowed to be selected.  
        Normally this constraint should always be enabled.  
    
    - **NonTrivialSolutionConstraint**  
        The purpose of this constraint is to avoid trivial solution (i.e. instances placed nowhere).  
        Normally this constraint should always be enabled.
    
    - **MaxRamAllocationPerHostConstraint**  
        Cap the virtual ram allocation of hosts.  
        The following option should be set in configuration when using this constraint:  
        ```ram_allocation_ratio = <a positive real number>``` (virtual-to-physical ram allocation ratio, if >1.0 then over-allocation is allowed.)  
    
    - **MaxDiskAllocationPerHostConstraint**  
        Cap the virtual disk allocation of hosts.  
        The following option should be set in configuration when using this constraint:  
        ```disk_allocation_ratio = <a positive real number>``` (virtual-to-physical disk allocation ratio, if >1.0 then over-allocation is allowed.)  
    
    - **MaxVcpuAllocationPerHostConstraint**  
        Cap the vcpu allocation of hosts.  
        The following option should be set in configuration when using this constraint:  
        ```cpu_allocation_ratio = <a positive real number>``` (virtual-to-physical cpu allocation ratio, if >1.0 then over-allocation is allowed.)  
    
    - **MaxInstancesPerHostConstraint**  
        Specify the maximum number of instances placed in each host in each scheduling process.  
        The following scheduler hint is expected when using this constraint:  
        ```max_instances_per_host = <a positive integer>```  
        By default, max_instances_per_host = 1, resulting in an anti-affinity placement solution.  
    
    - **DifferentHostConstraint**  
        Force instances to be placed at different hosts as specified instance(s).  
        The following scheduler hint is expected when using this constraint:  
        ```different_host = <a (list of) instance uuid(s)>```  
    
    - **SameHostConstraint**  
        Force instances to be placed at same hosts as specified instance(s).  
        The following scheduler hint is expected when using this constraint:  
        ```same_host = <a (list) of instance uuid(s)>```  
    
    - **AvailablilityZoneConstraint**  
        Select hosts belongong to an availability zone.  
        The following option should be set in configuration when using this constraint:  
        ```default_availability_zone = <availability zone>```  
    
    - **IoOpsConstraint**  
        Ensure the concurrent I/O operations number of selected hosts are within a threshold.  
        The following option should be set in configuration when using this constraint:  
        ```max_io_ops_per_host = <a positive number>```

Examples
--------

This is an example usage for creating VMs with volume affinity using the solver scheduler.

* Install the solver scheduler.

* Update the nova.conf with following options:
{'fraction_non_alphanumeric': 0.07628757250834944, 'fraction_numerical': 0.0033397785199507824, 'mean_word_length': 3.535742758437417, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 28, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4371679', 'n_tokens_mistral': 4602, 'n_tokens_neox': 4411, 'n_words': 1732}