<commit_before>= Fire & Forget

Fire & Forget is a simple, framework agnostic, background task launcher for Ruby web apps.

What it does:

- Designed for infrequent calls to very long running operations
- Provides a mechanism for calling any external script from within a HTTP request
- Gives simple inter-process communication for status updates
- Launches long running tasks as completely independent processes so that you can restart/redeploy your application without interrupting any running tasks
- Provides a 'kill' mechanism so you can cancel your tasks

What it doesn't:

- Have any kind of queue mechanism. If you fire the same task twice then that task will be running twice
- Have any persistant state

== Quick Start

  gem install fire_and_forget

FAF works by running a simple TCP socket server so before we want to use it we
have to start this server.

  $ fire_forget
  FAF process 16235 listening on 127.0.0.1:3001...

The service defaults to listening to port 3001 on localhost. You can change
these values thus:

  $ fire_forget -a 192.168.1.22 -p 9090
  FAF process 16240 listening on 192.168.1.22:9090...

If you do this you will need to set both the web-app and the tasks to use the
right values using

  FireAndForget.port = 9090
  FireAndForget.bind_address = "192.168.1.22"

Now inside our Ruby app:

  require 'rubygems'
  require 'fire_and_forget'

  # First set register our task by giving it a name and a path to a script file
  FireAndForget.add_task(:long_running_task, "/path/to/script")

  # when we want to call the script we simple call #fire passing the name of the
  # task and the options we want to pass
  FireAndForget.fire(:long_running_task, {:param3 => "value3"})

  # Or, use the name as a method:
  FireAndForget.long_running_task({:param3 => "value3"})

This will result in the following command being exec'd in an independent process:

  /path/to/script --param3="value3"

It up to the script to parse and deal with the command line options.

You can pass a couple of extra options to the #add_task method for example:

  FireAndForget.add_task(:long_running_task, "/path/to/script", {:param1 => "value1", :param2 => "value2", :param3 => "value3"}, 12)

this will add some default parameters to all invocations of the task, so now the call

  FireAndForget.fire(:long_running_task, {:param3 => "newvalue3"})

will issue the following command in the background:

  /path/to/script --param1="value1" --param2="value2" --param3="newvalue3"

Parameter values are JSON encoded so you can pass arrays or hashes. Again, it is
up to the script to decode these values.

The final parameter in the #add_task call is a "niceness" value. This will set the
UNIX nice setting of the script to the desired value so that you can launch
intensive background tasks without affecting the performance of your front line
HTTP services.

Interprocess communication is relatively easy. Take the following as the source of the script "/path/to/script"

  #!/usr/bin/env ruby

  require 'rubygems'
  require 'fire_and_forget'

  # this will mix in the comms methods and map this task to the :long_running_task label
  # used in the calling script
  include FireAndForget::Daemon[:long_running_task]

  30.times do |i|
    # update our status. What you put here is up to you, but it should be a String
    set_task_status("#{i+1} of 30")
    sleep(1)
  end

Now in the client all we have to do to get the status for our task is:

  FireAndForget.get_status(:long_running_task)
  # => "18 of 30"

If we decided we've had enough then we can kill it:

  # Send "SIG_TERM" to our task
  FireAndForget.term(:long_running_task)

  # Or send any signal (see the Process.kill documentation)
  FireAndForget.kill(:long_running_task, "HUP")

== Security

F&F is intended to be run in a relatively trusted environment and by default only
binds to localhost to stop external access. But to stop unintended access to
system commands, only scripts belonging to the same user as the F&F server
process are executed. This might not be enough so in the future I might add a
limit on the locations of runnable scripts.

== TODO

- Embed info about the F&F server address into the script parameters and then silently extract those inside the Daemon module. Otherwise you might have to duplicate configuration values (or jump through hoops to avoid it)
- Improve security restrictions
- Improve the tests so they are closer to testing the full stack

== Contributing to fire_and_forget

* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet<commit_msg>updated readme for latest socket/permissions changes<commit_after>= Fire & Forget

Fire & Forget is a simple, framework agnostic, background task launcher for Ruby web apps.

What it does:

- Designed for infrequent calls to very long running operations
- Provides a mechanism for calling any external script from within a HTTP request
- Gives simple inter-process communication for status updates
- Launches long running tasks as completely independent processes so that you can restart/redeploy your application without interrupting any running tasks
- Provides a 'kill' mechanism so you can cancel your tasks

What it doesn't:

- Have any kind of queue mechanism. If you fire the same task twice then that task will be running twice
- Have any persistant state
- Work in non-UNIX environments

== Quick Start

  gem install fire_and_forget

FAF works by running a simple UNIX socket server so before we want to use it we
have to start this server.

  $ fire_forget
  FAF process 16235 listening on /tmp/fire_and_forget.sock ...

To change the path of the socket file use the '-s' parameter:

  $ fire_forget -s /path/to/socket
  FAF process 16240 listening on /path/to/socket ...

If you do this you will need to set the web-app to use the right values:

  FireAndForget.socket = "/path/to/socket"

Now inside our Ruby app:

  require 'rubygems'
  require 'fire_and_forget'

  # First set register our task by giving it a name and a path to a script file
  FireAndForget.add_task(:long_running_task, "/path/to/script")
  # => #<FireAndForget::TaskDescription @name=:long_running_task ...>

  # when we want to call the script we simple call #fire passing the name of the
  # task and the options we want to pass
  FireAndForget.fire(:long_running_task, {:param3 => "value3"})
  # => "OK"

  # Or, use the name as a method:
  FireAndForget.long_running_task({:param3 => "value3"})
  # => "OK"

This will result in the following command being exec'd in an independent process:

  /path/to/script --param3="value3"

It up to the script to parse and deal with the command line options.

You can pass a couple of extra options to the #add_task method for example:

  FireAndForget.add_task(:long_running_task, "/path/to/script", {:param1 => "value1", :param2 => "value2", :param3 => "value3"}, 12)

this will add some default parameters to all invocations of the task, so now the call

  FireAndForget.fire(:long_running_task, {:param3 => "newvalue3"})

will issue the following command in the background:

  /path/to/script --param1="value1" --param2="value2" --param3="newvalue3"

Parameter values are JSON encoded so you can pass arrays or hashes. Again, it is
up to the script to decode these values.

The final parameter in the #add_task call is a "niceness" value. This will set the
UNIX nice setting of the script to the desired value so that you can launch
intensive background tasks without affecting the performance of your front line
HTTP services.

Interprocess communication is relatively easy. Take the following as the source of the script "/path/to/script"

  #!/usr/bin/env ruby

  require 'rubygems'
  require 'fire_and_forget'

  # this will mix in the F&F status methods
  # the F&F configuration (socket and task-name) is calculated automatically from ENV parameters
  include FireAndForget::Daemon

  30.times do |i|
    # update our status. What you put here is up to you, but it should be a String
    set_task_status("#{i+1} of 30")
    sleep(1)
  end

Now in the client all we have to do to get the status for our task is:

  FireAndForget.get_status(:long_running_task)
  # => "18 of 30"

If we decided we've had enough then we can kill it:

  # Send "SIG_TERM" to our task
  FireAndForget.term(:long_running_task)

  # Or send any signal (see the Process.kill documentation)
  FireAndForget.kill(:long_running_task, "HUP")

== Supporting multiple users

Tasks will attempt to run as the same user as the originating process. To enable
this, and hence enable a single F&F server to launch tasks for any number of
separate web-apps, simply run the F&F server as root. See the following section on
Security for the implications of this and how to stop unlimited access to the service.

== Security

F&F is intended to be run in a relatively trusted environment. You the UNIX
socket file has its permissions set so that only the process owner and a defined
group can write to the file. To change the UNIX group that has access pass the
group name as a parameter to the server:

  $ fire_forget -s /path/to/socket -g webapp_group

Only members of the 'webapp_group' will be able to launch processes through
the server. Additionally, only scripts belonging to the same user as the F&F task description
are executed.

In the meantime, as I say, this is defined to be run in a trusted environment, for instance on a
single non-shared server (potentially running many sites).

== TODO

- Improve security restrictions (perhaps by limiting the tasks that each user can launch)
- Improve the tests so they are closer to testing the full stack

== Contributing to fire_and_forget

* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
{'fraction_non_alphanumeric': 0.06193338086991953, 'fraction_numerical': 0.013955383518386473, 'mean_word_length': 4.060824742268041, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2035432', 'n_tokens_mistral': 2851, 'n_tokens_neox': 2685, 'n_words': 1406}