<commit_before># workerholic
A Background Job Processing Manager

## Installation
### Installing the gem
Install the gem with the following command:

    $ gem install workerholic

Or, add the following line to your application's Gemfile:

    gem 'workerholic', '~> 0.1'

And then make sure to execute the following command:

    $ bundle install

### Starting Redis
In order to start Redis, execute the following:

    redis-server

By default, Workerholic will try to connect to Redis via `localhost:6379`.

For a production environment, make sure to set a `REDIS_URL` environment variable to the address of your redis server.

## Usage
### Including Workerholic

In order to perform your jobs asynchronously you will need to include Workerholic in your job classes:

```ruby
class MyJob
  include Workerholic::Job

  def perform(args)
    # job logic goes here
  end
end
```

It is important to follow this pattern:
- include `Workerholic::Job` in your job class
- define a `perform` method holding your job's logic

### Job Options
#### Specifying a Queue

Workerholic allows you to specify a name for the queue that your job will be enqueued in:

```ruby
class MyJob
  include Workerholic::Job

  job_options queue_name: 'my_queue'

  def perform(args)
    # job logic goes here
  end
end
```

#### Performing a Job Asynchronously

You can perform a job asynchronously:

```ruby
my_job = MyJob.new
my_job.perform_async(arg1, arg2)
```

This will ensure that your job is performed in the background, asynchronously, as soon as possible.

#### Scheduling the Job

You can schedule a job to be executed at a later time:

```ruby
my_job = MyJob.new
my_job.peform_delayed(100, arg1, arg2)
```

The above example ensures that `my_job` will be performed in 100 seconds.

## Configuration
### Loading Application's Files
#### For a Rails Application

When using Workerholic with a Rails application, as long as you make sure to start Workerholic from the root directory of your Rails application, it will automatically detect your Rails application and eager load your application's files.

#### For Another Application

For Workerholic to execute the jobs you enqueue properly it needs to have access to the job classes.

Make sure to require all classes/dependencies needed with a single file:

    workerholic -r my_app/all_dependencies_needed.rb

### Setting Number of Workers

When starting Workerholic you can specify the number of workers you want running and performing jobs:

    workerholic -w 25

### Setting Number of Redis Connections

Internally, Workerholic uses a connection pool. By default, Workerholic will create a connection pool with a number of `workers_count + 5` Redis connections.

In a production environment, you might be limited by the number of concurrent connections to Redis you are allowed to have. Make sure to check what the limit is and you can then start Workerholic by specifying a number of connections:

    workerholic -c 10

### Setting Number of Processes to Boot

Workerholic allows you to start multiple processes in parallel by forking children processes from the main process. This can be achieved with the following option:

    workerholic -p 3

This will allow you to run 3 processes in parallel, with each process having its own workers and connection pool to Redis.

## Integration
### ActiveJob

Workerholic integrates with ActiveJob. Add the following line to your `application.rb` file:

```ruby
class Application < Rails::Application
  # ...

  config.active_job.queue_adapter = :workerholic<commit_msg>add auto-balancing option in Readme<commit_after># workerholic
A Background Job Processing Manager

## Installation
### Installing the gem
Install the gem with the following command:

    $ gem install workerholic

Or, add the following line to your application's Gemfile:

    gem 'workerholic', '~> 0.1'

And then make sure to execute the following command:

    $ bundle install

### Starting Redis
In order to start Redis, execute the following:

    redis-server

By default, Workerholic will try to connect to Redis via `localhost:6379`.

For a production environment, make sure to set a `REDIS_URL` environment variable to the address of your redis server.

## Usage
### Including Workerholic

In order to perform your jobs asynchronously you will need to include Workerholic in your job classes:

```ruby
class MyJob
  include Workerholic::Job

  def perform(args)
    # job logic goes here
  end
end
```

It is important to follow this pattern:
- include `Workerholic::Job` in your job class
- define a `perform` method holding your job's logic

### Job Options
#### Specifying a Queue

Workerholic allows you to specify a name for the queue that your job will be enqueued in:

```ruby
class MyJob
  include Workerholic::Job

  job_options queue_name: 'my_queue'

  def perform(args)
    # job logic goes here
  end
end
```

### Performing a Job Asynchronously

You can perform a job asynchronously:

```ruby
my_job = MyJob.new
my_job.perform_async(arg1, arg2)
```

This will ensure that your job is performed in the background, asynchronously, as soon as possible.

### Scheduling the Job

You can schedule a job to be executed at a later time:

```ruby
my_job = MyJob.new
my_job.peform_delayed(100, arg1, arg2)
```

The above example ensures that `my_job` will be performed in 100 seconds.

## Configuration
### Loading Application's Files
#### For a Rails Application

When using Workerholic with a Rails application, as long as you make sure to start Workerholic from the root directory of your Rails application, it will automatically detect your Rails application and eager load your application's files.

#### For Another Application

For Workerholic to execute the jobs you enqueue properly it needs to have access to the job classes.

Make sure to require all classes/dependencies needed with a single file:

    workerholic -r my_app/all_dependencies_needed.rb

### Setting Number of Workers

When starting Workerholic you can specify the number of workers you want running and performing jobs:

    workerholic -w 25

### Setting Number of Redis Connections

Internally, Workerholic uses a connection pool. By default, Workerholic will create a connection pool with a number of `workers_count + 5` Redis connections.

In a production environment, you might be limited by the number of concurrent connections to Redis you are allowed to have. Make sure to check what the limit is and you can then start Workerholic by specifying a number of connections:

    workerholic -c 10

### Setting Number of Processes to Boot

Workerholic allows you to start multiple processes in parallel by forking children processes from the main process. This can be achieved with the following option:

    workerholic -p 3

This will allow you to run 3 processes in parallel, with each process having its own workers and connection pool to Redis.

### Enabling Workers Auto-Balancing Option

By default, Workerholic will evenly provision workers between job queues.
Use the following option if you want to have workers provisioned based on the load for each queue:

    workerholic -a

This will ensure that each queue will be provisioned with a number of workers based on its relative load compared to the aggregated load for all job queues.

## Integration
### ActiveJob

Workerholic integrates with ActiveJob. Add the following line to your `application.rb` file:

```ruby
class Application < Rails::Application
  # ...

  config.active_job.queue_adapter = :workerholic
{'fraction_non_alphanumeric': 0.055245683930942896, 'fraction_numerical': 0.006108897742363878, 'mean_word_length': 4.329794762915782, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3968782', 'n_tokens_mistral': 2093, 'n_tokens_neox': 1958, 'n_words': 1047}