<commit_before>.. meta::
    :author: Cask Data, Inc.
    :copyright: Copyright © 2014-2015 Cask Data, Inc.

.. _workflows:

============================================
Workflows
============================================

**Workflows** are used to execute a series of :ref:`MapReduce <mapreduce>` or :ref:`Spark <spark>`.

A Workflow is given a sequence of jobs that follow each other, with an optional schedule
to run the Workflow periodically. Upon successful execution of a job, the control is
transferred to the next job in sequence until the last job in the sequence is executed. Upon
failure, the execution is stopped at the failed job and no subsequent jobs in the sequence
are executed.

To execute MapReduce or Spark programs in a Workflow, you will need to add them in your
application along with the Workflow. You can optionally add a schedule (such as a
`crontab schedule <../../reference-manual/javadocs/co/cask/cdap/api/app/AbstractApplication.html#scheduleWorkflow(java.lang.String,%20java.lang.String,%20java.lang.String)>`__)
to the Workflow::

  public void configure() {
    ...
    addMapReduce(new MyMapReduce());
    addMapReduce(new AnotherMapReduce());
    addSpark(new MySpark());
    addWorkflow(new MyWorkflow());
    scheduleWorkflow("FiveHourSchedule", "0 */5 * * *", "MyWorkflow");
    ...
  }

You'll then extend the ``AbstractWorkflow`` class and implement the
``configure()`` method. Inside ``configure``, you can add multiple MapReduce, Spark programs or custom actions
to the Workflow. The programs will be executed in the order they are specified in the ``configure``
method::

  public static class MyWorkflow extends AbstractWorkflow {

    @Override
    public void configure() {
        setName("MyWorkflow");
        setDescription("MyWorkflow description");
        addMapReduce("MyMapReduce");
        addSpark("MySpark");
        addMapReduce("AnotherMapReduce");
        addAction(new MyAction());
    }
  }

In this example, the ``MyWorkflow`` will be executed every 5 hours. During each execution of the Workflow,
the ``MyMapReduce``, ``MySpark``, and ``AnotherMapReduce`` programs and the ``MyAction`` custom action will be executed in order.

.. rubric::  Workflow Custom Action

In addition to MapReduce and Spark programs, Workflow can also execute custom actions. Custom actions are
implemented in Java and can perform tasks such as sending an email. To define
custom action, you will need to extend the ``AbstractWorkflowAction`` and implement the ``run()`` method::

  public static class MyAction extends AbstractWorkflowAction {

    @Override
    public void run() {
      // your code goes here
    }
  }

The custom action then can be added to the Workflow using the ``addAction()`` method as shown above.

.. rubric::  Example of Using a Workflow

- For an example of use of **a Workflow,** see the :ref:`Purchase
  <examples-purchase>` example.<commit_msg>Added documentation for the Workflow concurrent runs.<commit_after>.. meta::
    :author: Cask Data, Inc.
    :copyright: Copyright © 2014-2015 Cask Data, Inc.

.. _workflows:

============================================
Workflows
============================================

**Workflows** are used to execute a series of :ref:`MapReduce <mapreduce>` or :ref:`Spark <spark>`.

A Workflow is given a sequence of jobs that follow each other, with an optional schedule
to run the Workflow periodically. Upon successful execution of a job, the control is
transferred to the next job in sequence until the last job in the sequence is executed. Upon
failure, the execution is stopped at the failed job and no subsequent jobs in the sequence
are executed.

To execute MapReduce or Spark programs in a Workflow, you will need to add them in your
application along with the Workflow. You can optionally add a schedule (such as a
`crontab schedule <../../reference-manual/javadocs/co/cask/cdap/api/app/AbstractApplication.html#scheduleWorkflow(java.lang.String,%20java.lang.String,%20java.lang.String)>`__)
to the Workflow::

  public void configure() {
    ...
    addMapReduce(new MyMapReduce());
    addMapReduce(new AnotherMapReduce());
    addSpark(new MySpark());
    addWorkflow(new MyWorkflow());
    scheduleWorkflow("FiveHourSchedule", "0 */5 * * *", "MyWorkflow");
    ...
  }

You'll then extend the ``AbstractWorkflow`` class and implement the
``configure()`` method. Inside ``configure``, you can add multiple MapReduce, Spark programs or custom actions
to the Workflow. The programs will be executed in the order they are specified in the ``configure``
method::

  public static class MyWorkflow extends AbstractWorkflow {

    @Override
    public void configure() {
        setName("MyWorkflow");
        setDescription("MyWorkflow description");
        addMapReduce("MyMapReduce");
        addSpark("MySpark");
        addMapReduce("AnotherMapReduce");
        addAction(new MyAction());
    }
  }

In this example, the ``MyWorkflow`` will be executed every 5 hours. During each execution of the Workflow,
the ``MyMapReduce``, ``MySpark``, and ``AnotherMapReduce`` programs and the ``MyAction`` custom action will be executed in order.

.. rubric::  Workflow Custom Action

In addition to MapReduce and Spark programs, Workflow can also execute custom actions. Custom actions are
implemented in Java and can perform tasks such as sending an email. To define
custom action, you will need to extend the ``AbstractWorkflowAction`` and implement the ``run()`` method::

  public static class MyAction extends AbstractWorkflowAction {

    @Override
    public void run() {
      // your code goes here
    }
  }

The custom action then can be added to the Workflow using the ``addAction()`` method as shown above.

.. rubric::  Concurrent Workflows

By default, Workflow runs sequentially. Multiple instances of the Workflow can be run concurrently. To enable the
concurrent runs for the Workflow, set its runtime argument ``concurrent.runs.enabled`` to ``true``.

.. rubric::  Example of Using a Workflow

- For an example of use of **a Workflow,** see the :ref:`Purchase
  <examples-purchase>` example.
{'fraction_non_alphanumeric': 0.11969993476842791, 'fraction_numerical': 0.004892367906066536, 'mean_word_length': 4.246364414029085, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5661977', 'n_tokens_mistral': 1681, 'n_tokens_neox': 1605, 'n_words': 707}