<commit_before># fastly_nsq [![Build Status](https://travis-ci.org/fastly/fastly_nsq.svg?branch=master)](https://travis-ci.org/fastly/fastly_nsq)

*NOTE: This is a point-release
which is not yet suitable for production.
Use at your own peril.*

NSQ adapter and testing objects
for using the NSQ messaging system
in your Ruby project.

This library is intended
to facilitate publishing and consuming
messages on an NSQ messaging queue.

We also include fakes
to make testing easier.

This library is dependent
on the [`nsq-ruby`] gem.

[`nsq-ruby`]: https://github.com/wistia/nsq-ruby

Please use [GitHub Issues] to report bugs.

[GitHub Issues]: https://github.com/fastly/fastly_nsq/issues


## Install

`fastly_nsq` is a Ruby Gem
tested against Rails `>= 4.2`
and Ruby `>= 2.1.8`.

To get started,
add `fastly_nsq` to your `Gemfile`
and `bundle install`.

## Usage

### `FastlyNsq::Producer`

This is a class
which provides an adapter to the
fake and real NSQ producers.
These are used to
write messages onto the queue:

```ruby
message_data = {
  "data" => {
    "key" => "value"
  }
}

producer = FastlyNsq::Producer.new(
  nsqd: ENV.fetch('NSQD_TCP_ADDRESS'),
  topic: topic,
)

producer.write(message_data.to_json)
```
The mock/real strategy used
can be switched
by adding an environment variable
to your application:

```ruby
# for the fake
ENV['FAKE_QUEUE'] = true

# for the real thing
ENV['FAKE_QUEUE'] = false
```

### `FastlyNsq::Consumer`
This is a class
which provides an adapter to the
fake and real NSQ consumers.
These are used to
read messages off of the queue:

```ruby
consumer = FastlyNsq::Consumer.new(
  topic: 'topic',
  channel: 'channel'
)

consumer.size #=> 1
message = consumer.pop
message.body #=> "{ 'data': { 'key': 'value' } }"
message.finish
consumer.size #=> 0
consumer.terminate
```

As above,
the mock/real strategy used
can be switched by setting the
`FAKE_QUEUE` environment variable appropriately.

### `FastlyNsq::Listener`

To process the next message on the queue:

```ruby
topic     = 'user_created'
channel   = 'my_consuming_service'
processor = MessageProcessor

FastlyNsq::Listener.new(topic: topic, channel: channel, processor: processor).go(run_once: true)
```

This will pop the next message
off of the queue
and send the JSON text body
to `MessageProcessor.process(message_body, topic)`.

To initiate a blocking loop to process messages continuously:

```ruby
topic     = 'user_created'
channel   = 'my_consuming_service'
processor = MessageProcessor

FastlyNsq::Listener.new(topic: topic, channel: channel, processor: processor).go
```

This will block until
there is a new message on the queue,
      pop the next message
      off of the queue
      and send it to `MessageProcessor.process(message_body, topic)`.

### `FastlyNsq::RakeTask`

To help facilitate running the `FastlyNsq::Listener` in a blocking fashion
outside your application, a simple `RakeTask` is provided.

The task will listen
to all specified topics,
each in a separate thread.

This task can be added into your `Rakefile` in one of two ways:

Using a block:
```ruby
require 'fastly_nsq/rake_task'

FastlyNsq::RakeTask.new(:listen_task) do |task|
  task.channel = 'some_channel'
  task.topics  = {
    'some_topic' => SomeMessageProcessor
  }
end
```

The task can also define a `call`-able "preprocessor" (called before any `Processor.process`) and a custom `logger`.

See the [`Rakefile`](examples/Rakefile) file
for more detail.

### `FastlyNsq::CLI`

To help facilitate running the `FastlyNsq::Listener` in a blocking fashion
outside your application, a `CLI` and bin script [`fastly_nsq`](bin/fastly_nsq)
are provided.

This can be setup ahead of time by calling `FastlyNsq.configure` and passing
block. An exmaple of this can be found here: [`Example Config`](exmaple_config_class.rb)

An example of using the cli:
```bash
./bin/fastly_nsq -r ./example_config_class.rb -L ./test.log -P ./fastly_nsq.pid -v -d -t 4
```

### `FastlyNsq::Messenger`

Wrapper around a producer for sending messages and persisting producer objects.

```ruby
FastlyNsq::Messenger.deliver(message: msg, on_topic: 'my_topic', originating_service: 'my service')
```

This will use a FastlyNsq::Producer for the given topic or create on if it isn't<commit_msg>Update documentation.<commit_after># fastly_nsq [![Build Status](https://travis-ci.org/fastly/fastly_nsq.svg?branch=master)](https://travis-ci.org/fastly/fastly_nsq)

*NOTE: This is a point-release
which is not yet suitable for production.
Use at your own peril.*

NSQ adapter and testing objects
for using the NSQ messaging system
in your Ruby project.

This library is intended
to facilitate publishing and consuming
messages on an NSQ messaging queue.

We also include fakes
to make testing easier.

This library is dependent
on the [`nsq-ruby`] gem.

[`nsq-ruby`]: https://github.com/wistia/nsq-ruby

Please use [GitHub Issues] to report bugs.

[GitHub Issues]: https://github.com/fastly/fastly_nsq/issues


## Install

`fastly_nsq` is a Ruby Gem
tested against Rails `>= 4.2`
and Ruby `>= 2.1.8`.

To get started,
add `fastly_nsq` to your `Gemfile`
and `bundle install`.

## Usage

### `FastlyNsq::Producer`

This is a class
which provides an adapter to the
fake and real NSQ producers.
These are used to
write messages onto the queue:

```ruby
message_data = {
  "data" => {
    "key" => "value"
  }
}

producer = FastlyNsq::Producer.new(
  nsqd: ENV.fetch('NSQD_TCP_ADDRESS'),
  topic: topic,
)

producer.write(message_data.to_json)
```
The mock/real strategy used
can be switched
by adding an environment variable
to your application:

```ruby
# for the fake
ENV['FAKE_QUEUE'] = true

# for the real thing
ENV['FAKE_QUEUE'] = false
```

### `FastlyNsq::Consumer`
This is a class
which provides an adapter to the
fake and real NSQ consumers.
These are used to
read messages off of the queue:

```ruby
consumer = FastlyNsq::Consumer.new(
  topic: 'topic',
  channel: 'channel'
)

consumer.size #=> 1
message = consumer.pop
message.body #=> "{ 'data': { 'key': 'value' } }"
message.finish
consumer.size #=> 0
consumer.terminate
```

As above,
the mock/real strategy used
can be switched by setting the
`FAKE_QUEUE` environment variable appropriately.

### `FastlyNsq::Listener`

To process the next message on the queue:

```ruby
topic     = 'user_created'
channel   = 'my_consuming_service'
processor = MessageProcessor

FastlyNsq::Listener.new(topic: topic, channel: channel, processor: processor).go(run_once: true)
```

This will pop the next message
off of the queue
and send the JSON text body
to `MessageProcessor.process(message_body, topic)`.

To initiate a blocking loop to process messages continuously:

```ruby
topic     = 'user_created'
channel   = 'my_consuming_service'
processor = MessageProcessor

FastlyNsq::Listener.new(topic: topic, channel: channel, processor: processor).go
```

This will block until
there is a new message on the queue,
      pop the next message
      off of the queue
      and send it to `MessageProcessor.process(message_body, topic)`.

### `FastlyNsq::RakeTask`

To help facilitate running the `FastlyNsq::Listener` in a blocking fashion
outside your application, a simple `RakeTask` is provided.

The task will listen
to all specified topics,
each in a separate thread.

This task can be added into your `Rakefile` in one of two ways:

Using a block:
```ruby
require 'fastly_nsq/rake_task'

FastlyNsq::RakeTask.new(:listen_task) do |task|
  task.channel = 'some_channel'
  task.topics  = {
    'some_topic' => SomeMessageProcessor
  }
end
```

The task can also define a `call`-able "preprocessor" (called before any `Processor.process`) and a custom `logger`.

See the [`Rakefile`](examples/Rakefile) file
for more detail.

### `FastlyNsq::CLI`

To help facilitate running the `FastlyNsq::Listener` in a blocking fashion
outside your application, a `CLI` and bin script [`fastly_nsq`](bin/fastly_nsq)
are provided.

This can be setup ahead of time by calling `FastlyNsq.configure` and passing
block. An exmaple of this can be found here: [`Example Config`](exmaple_config_class.rb)

An example of using the cli:
```bash
./bin/fastly_nsq -r ./example_config_class.rb -L ./test.log -P ./fastly_nsq.pid -v -d -t 4
```

### `FastlyNsq::Messenger`

Wrapper around a producer for sending messages and persisting producer objects.

```ruby
FastlyNsq::Messenger.deliver(message: msg, on_topic: 'my_topic', originating_service: 'my service')
```

You can also optionally pass custom metadata.

```ruby
FastlyNsq::Messenger.deliver(message: msg, on_topic: 'my_topic', originating_service: 'my service', meta: { test: 'test' })
```

This will use a FastlyNsq::Producer for the given topic or create on if it isn't
{'fraction_non_alphanumeric': 0.11000229410415233, 'fraction_numerical': 0.0018352833218628125, 'mean_word_length': 4.654345006485085, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 19, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '843438', 'n_tokens_mistral': 2914, 'n_tokens_neox': 2789, 'n_words': 1018}