<commit_before>.. _chapter_tutorial_mpi_tasks:

*******************
MPI tasks
*******************

So far we have run a sequential tasks in a number of configurations.

This example introduces two new concepts: running multi-core MPI tasks
and specifying input data for the task, in this case a simple python MPI script.

----------------
Edit The Example
----------------

Download the file ``mpi_tasks.py`` with the following command:

.. code-block:: bash

    curl -O https://raw.githubusercontent.com/radical-cybertools/radical.pilot/readthedocs/examples/tutorial/mpi_tasks.py

Open the file ``mpi_tasks.py`` with your favorite editor. The example might
work right out of the box on your local machine, this depends whether you
have a local MPI installation. However, if you want to try it out with
different resources, like remote HPC clusters, look for the sections marked:

.. code-block:: python

        # ----- CHANGE THIS -- CHANGE THIS -- CHANGE THIS -- CHANGE THIS ------

and change the code below according to the instructions in the comments.


------------------------------
Download the Application Code
------------------------------

This example makes use of an application that we first download to our own
environment and then have staged as input to the MPI tasks.

Download the file ``helloworld_mpi.py`` with the following command:

.. code-block:: bash

    curl -O https://raw.githubusercontent.com/radical-cybertools/radical.pilot/readthedocs/examples/helloworld_mpi.py


-------------
Run the Code
-------------

** This assumes you have installed RADICAL-Pilot either globally or in a 
Python virtualenv. You also need access to a MongoDB server.**

Set the `RADICAL_PILOT_DBURL` environment variable in your shell to the 
MongoDB server you want to use, for example:

.. code-block:: bash
        
        export RADICAL_PILOT_DBURL=mongodb://192.168.127.12:27017/

If RADICAL-Pilot is installed and the MongoDB URL is set, you should be good
to run your program: 

.. code-block:: bash
<commit_msg>Rewording.<commit_after>.. _chapter_tutorial_mpi_tasks:

*******************
MPI tasks
*******************

So far we have run a sequential tasks in a number of configurations.

This example introduces two new concepts: running multi-core MPI tasks
and specifying input data for the task, in this case a simple python MPI script.

------------
Preparation
------------

Download the file ``mpi_tasks.py`` with the following command:

.. code-block:: bash

    curl -O https://raw.githubusercontent.com/radical-cybertools/radical.pilot/readthedocs/examples/tutorial/mpi_tasks.py

Open the file ``mpi_tasks.py`` with your favorite editor. The example might
work right out of the box on your local machine, this depends whether you
have a local MPI installation. However, if you want to try it out with
different resources, like remote HPC clusters, look for the sections marked:

.. code-block:: python

        # ----- CHANGE THIS -- CHANGE THIS -- CHANGE THIS -- CHANGE THIS ------

and change the code below according to the instructions in the comments.

This example makes use of an application that we first download to our own
environment and then have staged as input to the MPI tasks.

Download the file ``helloworld_mpi.py`` with the following command:

.. code-block:: bash

    curl -O https://raw.githubusercontent.com/radical-cybertools/radical.pilot/readthedocs/examples/helloworld_mpi.py


----------
Execution
----------

** This assumes you have installed RADICAL-Pilot either globally or in a 
Python virtualenv. You also need access to a MongoDB server.**

Set the `RADICAL_PILOT_DBURL` environment variable in your shell to the 
MongoDB server you want to use, for example:

.. code-block:: bash
        
        export RADICAL_PILOT_DBURL=mongodb://192.168.127.12:27017/

If RADICAL-Pilot is installed and the MongoDB URL is set, you should be good
to run your program: 

.. code-block:: bash

{'fraction_non_alphanumeric': 0.12750316856780736, 'fraction_numerical': 0.008111533586818757, 'mean_word_length': 3.1536842105263156, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5505950', 'n_tokens_mistral': 1164, 'n_tokens_neox': 1046, 'n_words': 468}