<commit_before># Sidekiq::Batching

Allows identical sidekiq jobs to be processed with a single background call.

Sponsored by [<NAME>](http://evilmartians.com)

Useful for:
* Grouping asynchronous API index calls into bulks for bulk updating/indexing.
* Periodical batch updating of recently changing database counters.

## Usage

Create a worker:

```ruby
class ElasticBulkIndexWorker
  include Sidekiq::Worker

  sidekiq_options(
    queue: :batched_by_size,
    batch_size: 30,           # Jobs will be combined to groups of 30 items
    batch_flush_interval: 60, # Combined jobs will be executed at least every 60 seconds
    retry: 5
  )

  def perform(group)
    client = Elasticsearch::Client.new
    client.bulk(body: group.flatten)
  end
end<commit_msg>Sponsored moved to correct location<commit_after># Sidekiq::Batching

Allows identical sidekiq jobs to be processed with a single background call.

Useful for:
* Grouping asynchronous API index calls into bulks for bulk updating/indexing.
* Periodical batch updating of recently changing database counters.

Sponsored by [<NAME>](http://evilmartians.com)

## Usage

Create a worker:

```ruby
class ElasticBulkIndexWorker
  include Sidekiq::Worker

  sidekiq_options(
    queue: :batched_by_size,
    batch_size: 30,           # Jobs will be combined to groups of 30 items
    batch_flush_interval: 60, # Combined jobs will be executed at least every 60 seconds
    retry: 5
  )

  def perform(group)
    client = Elasticsearch::Client.new
    client.bulk(body: group.flatten)
  end
end
{'fraction_non_alphanumeric': 0.06976744186046512, 'fraction_numerical': 0.011627906976744186, 'mean_word_length': 4.078688524590164, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1480214', 'n_tokens_mistral': 491, 'n_tokens_neox': 452, 'n_words': 167}