<commit_before>Introduction
============

This tutorial is an introduction to dispel4py. We will see how to write dispel4py PEs, how to connect them together to form a workflow and how this workflow is executed in different environments.

How to write a PE
=================

In this section we are going to implement our first PE.

First you need to decide what kind of processing the PE will do and what the data units are that it processes. In our example we are implementing a PE that decides if a number is divisible by another number. The PE is configured with this divisor and for each input data item it tests whether the number can be divided by this divisor. It returns the input data item if it is not divisible.


Create a PE class
-----------------

To start with we create a PE that does only very few things::

    from dispel4py.base import IterativePE

    class MyFirstPE(IterativePE):

        def __init__(self, divisor):
            IterativePE.__init__(self)
            self.divisor = divisor

In this case we extend the base class :py:class:`dispel4py.base.IterativePE` which defines one input and one output, which is exactly what we need. We pass the divisor as an initialisation parameter to the object which stores it.


Implement the processing method
-------------------------------<commit_msg>small update<commit_after>Introduction
============

This tutorial is an introduction to dispel4py. We will see how to write dispel4py PEs, how to connect them together to form a workflow and how this workflow is executed in different environments.

How to write a PE
=================

In this section we are going to implement our first PE.

First you need to decide what kind of processing the PE will do and what the data units are that it processes. In our example we are implementing a PE that decides if a number is divisible by another number. The PE is configured with this divisor and for each input data item it tests whether the number can be divided by this divisor. It sends the input data item to its output if it is not divisible.


Create a PE class
-----------------

To start with we create a PE that does only very few things::

    from dispel4py.base import IterativePE

    class MyFirstPE(IterativePE):

        def __init__(self, divisor):
            IterativePE.__init__(self)
            self.divisor = divisor

In this case we extend the base class :py:class:`dispel4py.base.IterativePE` which defines one input and one output, which is exactly what we need. We pass the divisor as an initialisation parameter to the object which stores it.


Implement the processing method
-------------------------------
{'fraction_non_alphanumeric': 0.0849943587814968, 'fraction_numerical': 0.003008649868371568, 'mean_word_length': 3.215530903328051, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2921399', 'n_tokens_mistral': 639, 'n_tokens_neox': 612, 'n_words': 392}