<commit_before>.. _tutorial-development:


Tutorial: development
=====================

This tutorial demonstrates some features of ERPpeek in the interactive
shell.

This tutorial explains how to take part in the development of
Magentoerpconnect. It will explain how to use the different pieces of
the ``Connector`` addon to synchronize records with Magento.

.. contents:: Sections:
   :local:
   :backlinks: top


Run a function on an Event
--------------------------

3 events are registered in the ``Connector`` addon:

* ``on_record_create``
* ``on_record_write``
* ``on_record_unlink``

If you need to create a new :py:class:`connector.event.Event`, please
refer to the ``Connector`` documentation.

When a function has to be run when an event is fired, it must be
registered on this event. Those functions are called ``Consumers``.

In ``magentoerpconnect/consumer.py``, some consumers are already
defined. You can add your one, it should be decorated by
:py:func:`magentoerpconnect.consumer.magento_consumer` and by the event
which has to fire it::

    @on_record_write(model_names=['my.model'])
    @magento_consumer
    def my_consumer(session, model_name, record_id, fields=None):
        print 'Yeah'

.. note:: The consumers always start with the arguments ``session`` and
          ``model_name``. The next arguments vary, but they are defined
          by the :py:class:`connector.event.Event`


Find the good class for a model
-------------------------------


Create an import
----------------

You'll need to work on at least 4 pieces of the connector:

* a Synchronizer (presumably 2, we'll see why soon)
* a Mapper
* a Backend Adapter
* a Binder

You will also need to create / change the OpenERP models.

.. note:: Keep that in mind: try to modify at least as possible the
          OpenERP models and classes.

The synchronizer will handle the flow of the synchronization. It will
get the data from Magento using the Backend Adapter, transform it using
the Mapper, and use the Binder(s) to search the relation(s) with other
imported records.

Why do we need 2 synchronizers? Because an import is generally done in 2
phases:

1. The first synchronizer searches the list of all the ids to import.
2. The second synchronizer imports all the ids atomically.

We'll see in details a simple import: customer groups.

Models
''''''

First, we'll create the model::

    class magento_res_partner_category(orm.Model):
        _name = 'magento.res.partner.category'
        _inherit = 'magento.binding'
        _inherits = {'res.partner.category': 'openerp_id'}

        _columns = {
            'openerp_id': fields.many2one('res.partner.category',
                                           string='Partner Category',
                                           required=True,
                                           ondelete='cascade'),
            'tax_class_id': fields.integer('Tax Class ID'),
        }

        _sql_constraints = [
            ('magento_uniq', 'unique(backend_id, magento_id)',
             'A partner tag with same ID on Magento already exists.'),
        ]

Observations:

* We do not change ``res.partner.category`` but create a
  ``magento.res.partner.category`` model instead.
* It `_inherit` from `magento.binding`
* It contains the links to the Magento backend, the category and the
  ID on Magento.
* This model stores the data related to one category and one Magento
  backend as well, so this data does not pollute the category and does
  not criss-cross when several backends are connected.
* It ``_inherits`` the ``res.partner.category`` so we can directly use
  this model for the imports and the exports without complications.

We need to add the field ``magento_bind_ids`` in
``res.partner.category`` to relate to the Magento Bindings::

    class res_partner_category(orm.Model):
        _inherit = 'res.partner.category'

        _columns = {
            'magento_bind_ids': fields.one2many(
                'magento.res.partner.category',
                'openerp_id',
                string='Magento Bindings',
                readonly=True),
        }

That's the only thing we need to change (besides the view) on the
OpenERP category!

.. note:: The name of the field ``magento_bind_ids`` is a convention.

Ok, we're done with the models. Now the synchronizations!

Batch Importer
''''''''''''''

The first Synchronizer, which get the full list of ids to import is
usually a subclass of
:py:class:`magentoerpconnect.unit.import_synchronizer.BatchImportSynchronizer`.

The customer groups are simple enough to use a generic class::

    @magento
    class DelayedBatchImport(BatchImportSynchronizer):
        """ Delay import of the records """
        _model_name = [
                'magento.res.partner.category',
                ]

        def _import_record(self, record):
            """ Delay the import of the records"""
            job.import_record.delay(self.session,
                                    self.model._name,
                                    self.backend_record.id,
                                    record)

Observations:

* Decorated by ``@magento``: this synchronizer will be available for all
  versions of Magento. Decorate for instance with ``@magento1700`` to
  activate it for Magento 1.7 only.
* ``_model_name``: the list of models allowed to use this synchronizer
* We just override the ``_import_record`` hook, the search has already
  be done in
  :py:class:`magentoerpconnect.unit.import_synchronizer.BatchImportSynchronizer`.
* ``import_record`` is a job to import a record from its ID.
* Delay the import of each record, a job will be created for each record id.
* This synchronization does not need any Binder nor Mapper, but does
  need a Backend Adapter to be able to speak with Magento.

So, let's implement the **Backend Adapter**.

Backend Adapter
'''''''''''''''

Most of the Magento objects can use the generic class
:py:class`magentoerpconnect.unit.backend_adapter.GenericAdapter`.
However, the ``search`` entry point is not implemented in the API for
customer groups.

We'll replace it using ``list`` and select only the ids::

    @magento
    class PartnerCategoryAdapter(GenericAdapter):
        _model_name = 'magento.res.partner.category'
        _magento_model = 'ol_customer_groups'

        def search(self, filters=None):
            """ Search records according to some criterias
            and returns a list of ids

            :rtype: list
            """
            with magentolib.API(self.magento.location,
                                self.magento.username,
                                self.magento.password) as api:
                return [int(row['customer_group_id']) for row
                           in api.call('%s.list' % self._magento_model,
                                       [filters] if filters else [{}])]
            return []

Observations:

* ``_magento_model`` is the first part of the entry points in the API
  (ie. ``ol_customer_groups.list``)
* Only the ``search`` method is overriden.

We have all the pieces for the first part of the synchronization, just
need to...

Call the Batch Import
'''''''''''''''''''''

This import will be called from the **Magento Backend**, we modify it in
``magentoerpconnect/magento_model.py`` and add a method (to add in the
view as well, I won't write the view code here)::

    def import_customer_groups(self, cr, uid, ids, context=None):
        if not hasattr(ids, '__iter__'):
            ids = [ids]
        session = connector.ConnectorSession(cr, uid, context=context)
        for backend_id in ids:
            job.import_batch.delay(session, 'magento.res.partner.category',
                                   backend_id)

        return True

Observations:

* Declare a :py:class:`connector.connector.ConnectorSession`.
* Delay the job ``import_batch``.

Overview on the jobs
''''''''''''''''''''

We used 2 jobs: ``import_record`` and ``import_batch``. These jobs are
already there so you don't need to write them, but we can have a look
on them to understand what they do::

    def _get_environment(session, model_name, backend_id):
        model = session.pool.get('magento.backend')
        backend_record = model.browse(session.cr,
                                      session.uid,
                                      backend_id,
                                      session.context)
        return connector.Environment(backend_record, session, model_name)


    @connector.job
    def import_batch(session, model_name, backend_id, filters=None):
        """ Prepare a batch import of records from Magento """
        env = _get_environment(session, model_name, backend_id)
        importer = env.get_connector_unit(BatchImportSynchronizer)
        importer.run(filters)


    @connector.job
    def import_record(session, model_name, backend_id, magento_id):
        """ Import a record from Magento """
        env = _get_environment(session, model_name, backend_id)
        importer = env.get_connector_unit(MagentoImportSynchronizer)
        importer.run(magento_id)

Observations:

* Decorated by :py:class:`connector.connector.queue.job.job`, allow to
  ``delay`` the function.
* We create a new environment and ask for the good importer, respectively
  for batch imports and record imports. The environment returns an
  instance of the importer to use.

At this point, if one click on the button to import the categories, the
batch import would run, generate one job for each category to import,
and then all these jobs would fail. We need to create the second
synchronizer, the mapper and the binder.

Record Importer
'''''''''''''''

The import of customer groups is so simple that they can use a generic
class
:py:class:`magentoerpconnect.unit.import_synchronizer.SimpleRecordImport`.
We just need to add the model in the ``_model_name`` attribute::

    @magento
    class SimpleRecordImport(MagentoImportSynchronizer):
        """ Import one Magento Website """
        _model_name = [
                'magento.website',
                'magento.store',
                'magento.storeview',
                'magento.res.partner.category',
            ]

However, most of the imports will be more complicated than that. You
will often need to create a new class for a model, where you will need
to use some of the hooks to change the behavior
(``_import_dependencies``, ``_after_import`` for example).
Refers to the importers already created in the module and to the base
class
:py:class:`magentoerpconnect.unit.import_synchronizer.MagentoImportSynchronizer`.

The synchronizer asks to the appropriate **Mapper** to transform the data
(in ``_map_data``). Here is how we'll create the **Mapper**.

Mapper
''''''

The mapper takes the record in input from Magento, and generates the
OpenERP record. (or the reverse for the export Mappers)

The mapper for the customer groups is as follows::

    @magento
    class PartnerCategoryImportMapper(connector.ImportMapper):
        _model_name = 'magento.res.partner.category'

        direct = [
                ('customer_group_code', 'name'),
                ('tax_class_id', 'tax_class_id'),
                ]

        @mapping
        def magento_id(self, record):
            return {'magento_id': record['customer_group_id']}

        @mapping
        def backend_id(self, record):
            return {'backend_id': self.backend_record.id}


Observations:

* Some mappings are in ``direct`` and some use a method with a
  ``@mapping`` decorator.
* Methods allow to have more complex mappings.

Binder
''''''

For the last piece of the construct, it will be an easy one, because
normally all the Magento Models will use the same Binder, the so called
:py:class:`magentoerpconnect.unit.binder.MagentoModelBinder`.

We just need to add our model in the ``_model_name`` attribute::

    @magento
    class MagentoModelBinder(MagentoBinder):
        """
        Bindings are done directly on the model
        """
        _model_name = [
                'magento.website',
                'magento.store',
                'magento.storeview',
                'magento.res.partner.category',
            ]

    [...]

<commit_msg>doc: import tutorial<commit_after>.. _tutorial-development:


Tutorial: development
=====================

This tutorial demonstrates some features of ERPpeek in the interactive
shell.

This tutorial explains how to take part in the development of
Magentoerpconnect. It will explain how to use the different pieces of
the ``Connector`` addon to synchronize records with Magento.

.. contents:: Sections:
   :local:
   :backlinks: top


Run a function on an Event
--------------------------

3 events are registered in the ``Connector`` addon:

* ``on_record_create``
* ``on_record_write``
* ``on_record_unlink``

If you need to create a new :py:class:`connector.event.Event`, please
refer to the ``Connector`` documentation.

When a function has to be run when an event is fired, it must be
registered on this event. Those functions are called ``Consumers``.

In ``magentoerpconnect/consumer.py``, some consumers are already
defined. You can add your one, it should be decorated by
:py:func:`magentoerpconnect.consumer.magento_consumer` and by the event
which has to fire it::

    @on_record_write(model_names=['my.model'])
    @magento_consumer
    def my_consumer(session, model_name, record_id, fields=None):
        print 'Yeah'

.. note:: The consumers always start with the arguments ``session`` and
          ``model_name``. The next arguments vary, but they are defined
          by the :py:class:`connector.event.Event`


Find the good 'unit' class for a model
--------------------------------------

Assume that you already have an ``Environment``.

.. note:: An ``Environment`` is the scope where the synchronizations are
          done. It contains the browse record of the backend
          (``magento.backend``), a ``ConnectorSession`` (container for
          ``cr``, ``uid``, ``context``) and the name of the model we are
          working with).

You can get an instance of the ``ConnectorUnit`` to use from the
environment.  You'll need to ask a connector unit with the base class
which interests you.  Say you want a Synchronizer which import records
from Magento::

    importer = environment.get_connector_unit(MagentoImportSynchronizer)

``importer`` is an instance of the importer to use for the model of the
environment.

Say you want a binder for your model::

    binder = environment.get_connector_unit(connector.Binder)

``binder`` is an instance of the binder for your model.

And so on...

.. note:: Every ``ConnectorUnit`` instance keeps the environment as
          attribute. It means that you can access to the environment
          from a synchronizer with ``self.environment``.


Create an import
----------------

You'll need to work on at least 4 pieces of the connector:

* a Synchronizer (presumably 2, we'll see why soon)
* a Mapper
* a Backend Adapter
* a Binder

You will also need to create / change the OpenERP models.

.. note:: Keep that in mind: try to modify at least as possible the
          OpenERP models and classes.

The synchronizer will handle the flow of the synchronization. It will
get the data from Magento using the Backend Adapter, transform it using
the Mapper, and use the Binder(s) to search the relation(s) with other
imported records.

Why do we need 2 synchronizers? Because an import is generally done in 2
phases:

1. The first synchronizer searches the list of all the ids to import.
2. The second synchronizer imports all the ids atomically.

We'll see in details a simple import: customer groups.

Models
''''''

First, we'll create the model::

    class magento_res_partner_category(orm.Model):
        _name = 'magento.res.partner.category'
        _inherit = 'magento.binding'
        _inherits = {'res.partner.category': 'openerp_id'}

        _columns = {
            'openerp_id': fields.many2one('res.partner.category',
                                           string='Partner Category',
                                           required=True,
                                           ondelete='cascade'),
            'tax_class_id': fields.integer('Tax Class ID'),
        }

        _sql_constraints = [
            ('magento_uniq', 'unique(backend_id, magento_id)',
             'A partner tag with same ID on Magento already exists.'),
        ]

Observations:

* We do not change ``res.partner.category`` but create a
  ``magento.res.partner.category`` model instead.
* It `_inherit` from `magento.binding`
* It contains the links to the Magento backend, the category and the
  ID on Magento.
* This model stores the data related to one category and one Magento
  backend as well, so this data does not pollute the category and does
  not criss-cross when several backends are connected.
* It ``_inherits`` the ``res.partner.category`` so we can directly use
  this model for the imports and the exports without complications.

We need to add the field ``magento_bind_ids`` in
``res.partner.category`` to relate to the Magento Bindings::

    class res_partner_category(orm.Model):
        _inherit = 'res.partner.category'

        _columns = {
            'magento_bind_ids': fields.one2many(
                'magento.res.partner.category',
                'openerp_id',
                string='Magento Bindings',
                readonly=True),
        }

That's the only thing we need to change (besides the view) on the
OpenERP category!

.. note:: The name of the field ``magento_bind_ids`` is a convention.

Ok, we're done with the models. Now the synchronizations!

Batch Importer
''''''''''''''

The first Synchronizer, which get the full list of ids to import is
usually a subclass of
:py:class:`magentoerpconnect.unit.import_synchronizer.BatchImportSynchronizer`.

The customer groups are simple enough to use a generic class::

    @magento
    class DelayedBatchImport(BatchImportSynchronizer):
        """ Delay import of the records """
        _model_name = [
                'magento.res.partner.category',
                ]

        def _import_record(self, record):
            """ Delay the import of the records"""
            job.import_record.delay(self.session,
                                    self.model._name,
                                    self.backend_record.id,
                                    record)

Observations:

* Decorated by ``@magento``: this synchronizer will be available for all
  versions of Magento. Decorate for instance with ``@magento1700`` to
  activate it for Magento 1.7 only.
* ``_model_name``: the list of models allowed to use this synchronizer
* We just override the ``_import_record`` hook, the search has already
  be done in
  :py:class:`magentoerpconnect.unit.import_synchronizer.BatchImportSynchronizer`.
* ``import_record`` is a job to import a record from its ID.
* Delay the import of each record, a job will be created for each record id.
* This synchronization does not need any Binder nor Mapper, but does
  need a Backend Adapter to be able to speak with Magento.

So, let's implement the **Backend Adapter**.

Backend Adapter
'''''''''''''''

Most of the Magento objects can use the generic class
:py:class`magentoerpconnect.unit.backend_adapter.GenericAdapter`.
However, the ``search`` entry point is not implemented in the API for
customer groups.

We'll replace it using ``list`` and select only the ids::

    @magento
    class PartnerCategoryAdapter(GenericAdapter):
        _model_name = 'magento.res.partner.category'
        _magento_model = 'ol_customer_groups'

        def search(self, filters=None):
            """ Search records according to some criterias
            and returns a list of ids

            :rtype: list
            """
            with magentolib.API(self.magento.location,
                                self.magento.username,
                                self.magento.password) as api:
                return [int(row['customer_group_id']) for row
                           in api.call('%s.list' % self._magento_model,
                                       [filters] if filters else [{}])]
            return []

Observations:

* ``_magento_model`` is the first part of the entry points in the API
  (ie. ``ol_customer_groups.list``)
* Only the ``search`` method is overriden.

We have all the pieces for the first part of the synchronization, just
need to...

Call the Batch Import
'''''''''''''''''''''

This import will be called from the **Magento Backend**, we modify it in
``magentoerpconnect/magento_model.py`` and add a method (to add in the
view as well, I won't write the view code here)::

    def import_customer_groups(self, cr, uid, ids, context=None):
        if not hasattr(ids, '__iter__'):
            ids = [ids]
        session = connector.ConnectorSession(cr, uid, context=context)
        for backend_id in ids:
            job.import_batch.delay(session, 'magento.res.partner.category',
                                   backend_id)

        return True

Observations:

* Declare a :py:class:`connector.connector.ConnectorSession`.
* Delay the job ``import_batch``.

Overview on the jobs
''''''''''''''''''''

We used 2 jobs: ``import_record`` and ``import_batch``. These jobs are
already there so you don't need to write them, but we can have a look
on them to understand what they do::

    def _get_environment(session, model_name, backend_id):
        model = session.pool.get('magento.backend')
        backend_record = model.browse(session.cr,
                                      session.uid,
                                      backend_id,
                                      session.context)
        return connector.Environment(backend_record, session, model_name)


    @connector.job
    def import_batch(session, model_name, backend_id, filters=None):
        """ Prepare a batch import of records from Magento """
        env = _get_environment(session, model_name, backend_id)
        importer = env.get_connector_unit(BatchImportSynchronizer)
        importer.run(filters)


    @connector.job
    def import_record(session, model_name, backend_id, magento_id):
        """ Import a record from Magento """
        env = _get_environment(session, model_name, backend_id)
        importer = env.get_connector_unit(MagentoImportSynchronizer)
        importer.run(magento_id)

Observations:

* Decorated by :py:class:`connector.connector.queue.job.job`, allow to
  ``delay`` the function.
* We create a new environment and ask for the good importer, respectively
  for batch imports and record imports. The environment returns an
  instance of the importer to use.

At this point, if one click on the button to import the categories, the
batch import would run, generate one job for each category to import,
and then all these jobs would fail. We need to create the second
synchronizer, the mapper and the binder.

Record Importer
'''''''''''''''

The import of customer groups is so simple that they can use a generic
class
:py:class:`magentoerpconnect.unit.import_synchronizer.SimpleRecordImport`.
We just need to add the model in the ``_model_name`` attribute::

    @magento
    class SimpleRecordImport(MagentoImportSynchronizer):
        """ Import one Magento Website """
        _model_name = [
                'magento.website',
                'magento.store',
                'magento.storeview',
                'magento.res.partner.category',
            ]

However, most of the imports will be more complicated than that. You
will often need to create a new class for a model, where you will need
to use some of the hooks to change the behavior
(``_import_dependencies``, ``_after_import`` for example).
Refers to the importers already created in the module and to the base
class
:py:class:`magentoerpconnect.unit.import_synchronizer.MagentoImportSynchronizer`.

The synchronizer asks to the appropriate **Mapper** to transform the data
(in ``_map_data``). Here is how we'll create the **Mapper**.

Mapper
''''''

The mapper takes the record in input from Magento, and generates the
OpenERP record. (or the reverse for the export Mappers)

The mapper for the customer groups is as follows::

    @magento
    class PartnerCategoryImportMapper(connector.ImportMapper):
        _model_name = 'magento.res.partner.category'

        direct = [
                ('customer_group_code', 'name'),
                ('tax_class_id', 'tax_class_id'),
                ]

        @mapping
        def magento_id(self, record):
            return {'magento_id': record['customer_group_id']}

        @mapping
        def backend_id(self, record):
            return {'backend_id': self.backend_record.id}


Observations:

* Some mappings are in ``direct`` and some use a method with a
  ``@mapping`` decorator.
* Methods allow to have more complex mappings.

Binder
''''''

For the last piece of the construct, it will be an easy one, because
normally all the Magento Models will use the same Binder, the so called
:py:class:`magentoerpconnect.unit.binder.MagentoModelBinder`.

We just need to add our model in the ``_model_name`` attribute::

    @magento
    class MagentoModelBinder(MagentoBinder):
        """
        Bindings are done directly on the model
        """
        _model_name = [
                'magento.website',
                'magento.store',
                'magento.storeview',
                'magento.res.partner.category',
            ]

    [...]


{'fraction_non_alphanumeric': 0.09506656225528583, 'fraction_numerical': 0.0012529365700861394, 'mean_word_length': 2.7715593620791497, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 28, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3826974', 'n_tokens_mistral': 7093, 'n_tokens_neox': 6945, 'n_words': 2659}