<commit_before>.. _staticanalysis/bot-project:

Project: staticanalysis/bot
===============================

:contact: `<NAME>`_, (backup `Release Management`_)

Static analysis bot is a Taskcluster task, triggered by *Pulse Listener* on every new Mozreview or Phabricator patch.
The task applies several code analyzers on each patch:

 * clang-tidy through ./mach static-analysis
 * clang-format through ./mach static-analysis
 * ./mach lint (mozlint)

And reports the results through several channels:

 * emails,
 * MozReview comments,
 * Phabricator reviews.

Developer setup
---------------

0. Requirements
"""""""""""""""

You'll need:

1. A `Taskcluster`_ account
2. Docker

1. Taskcluster secret
"""""""""""""""""""""

Once logged on Taskcluster, please check that you can view the contents of the Taskcluster secret : `repo:github.com/mozilla-releng/services:branch:master <https://tools.taskcluster.net/secrets/repo%3Agithub.com%2Fmozilla-releng%2Fservices%3Abranch%3Amaster>`_.

This secret holds the configuration for all the services, you can look at the ``staticanalysis/bot`` section for more details.

If you don't have access to that secret, or you need to make some changes to it, you can create a new secret under the "garbage" namespace (publicly visible to anyone); and use its name everywhere the ``master`` secret is mentioned in this documentation.

For example, you can create a secret called ``garbage/LOGIN/staticanalysis-bot-dev`` with this value:

.. code-block:: yaml

  common:
    APP_CHANNEL: master
  static-analysis-bot:
    ANALYZERS:
      - clang-tidy
      - clang-format
      - infer
      - mozlint
    PHABRICATOR:
      url: 'https://phabricator-dev.allizom.org/api/'
      api_key: api-YOURTOKEN
    REPORTERS:
      - reporter: mail
        emails:
          - <EMAIL>
      - reporter: phabricator

Just replace ``LOGIN`` with your username (e.g. ``michel``); ``api-YOURTOKEN`` with a new Conduit API Token from `Phabricator-dev https://phabricator-dev.allizom.org/settings/`_ (hint: on Bugzilla-dev, you can just "Sign in with GitHub"); and ``<EMAIL>`` with your email address.


2. Taskcluster client
"""""""""""""""""""""

You need to create a `Taskcluster client`_ to run the static analysis task on your computer.

Use the form to create a new client in your own namespace (the ``ClientId`` should be pre-filled with ``mozilla-auth0/ad|Mozilla-LDAP|login/``, simply add an explicit suffix, like ``static-analysis-dev``)

Add an explicit description, you can leave the ``Expires`` setting into the far future.

Add the Taskcluster scope needed to read the secret previously mentioned: ``secrets:get:repo:github.com/mozilla-releng/services:branch:master`` (or ``secrets:get:garbage/michel/staticanalysis-bot-dev`` if you're using your own secret)

You may also want to add the scope needed to send you an email report: ``notify:email:<EMAIL>`` (if your email is ``<EMAIL>``)

To summarize, you need to setup your client (if your login is ``michel``), like this:

============= ====================================================================
Key           Value
============= ====================================================================
ClientId      ``mozilla-auth0/ad|Mozilla-LDAP|michel/static-analysis-dev``
Description   My own static analysis dev. client
Client Scopes ``secrets:get:repo:github.com/mozilla-releng/services:branch:master``
              ``notify:email:<EMAIL>``
============= ====================================================================


.. warning::

  Save the **access token** provided by Taskcluster after creating your client, it won't be displayed afterwards


3. Project shell
""""""""""""""""

Run the following (where ``XXX`` is the Taskcluster access token):

.. code-block:: shell

  ./please shell staticanalysis/bot \
    --taskcluster-client-id="mozilla-auth0/ad|Mozilla-LDAP|michel/static-analysis-dev" \
    --taskcluster-access-token=XXX

Once the initial build finishes, you should get a green Nix shell, running in ``/app/src/staticanalysis/bot``.


4. Setup a Phabricator test
"""""""""""""""""""""""""""


.. note::
  Make sure your Taskcluster secret has a ``phabricator`` reporter setup, as follows (with a valid Phabricator uri & token):

  .. code-block:: yaml

    staticanalysis/bot:
      ...
      REPORTERS:
        - reporter: phabricator
          url: 'https://phabricator-dev.allizom.org/api/'
          api_key: api-XXXX


The bot needs an environment variable ``PHABRICATOR`` containing the PHID of the diff to be reviewed.

So you'll need to do the following in the nix shell:

.. code-block:: shell
<commit_msg>staticanlysis/bot: Update a phabricator configuration snippet. (#1569)<commit_after>.. _staticanalysis/bot-project:

Project: staticanalysis/bot
===============================

:contact: `<NAME>`_, (backup `Release Management`_)

Static analysis bot is a Taskcluster task, triggered by *Pulse Listener* on every new Mozreview or Phabricator patch.
The task applies several code analyzers on each patch:

 * clang-tidy through ./mach static-analysis
 * clang-format through ./mach static-analysis
 * ./mach lint (mozlint)

And reports the results through several channels:

 * emails,
 * MozReview comments,
 * Phabricator reviews.

Developer setup
---------------

0. Requirements
"""""""""""""""

You'll need:

1. A `Taskcluster`_ account
2. Docker

1. Taskcluster secret
"""""""""""""""""""""

Once logged on Taskcluster, please check that you can view the contents of the Taskcluster secret : `repo:github.com/mozilla-releng/services:branch:master <https://tools.taskcluster.net/secrets/repo%3Agithub.com%2Fmozilla-releng%2Fservices%3Abranch%3Amaster>`_.

This secret holds the configuration for all the services, you can look at the ``staticanalysis/bot`` section for more details.

If you don't have access to that secret, or you need to make some changes to it, you can create a new secret under the "garbage" namespace (publicly visible to anyone); and use its name everywhere the ``master`` secret is mentioned in this documentation.

For example, you can create a secret called ``garbage/LOGIN/staticanalysis-bot-dev`` with this value:

.. code-block:: yaml

  common:
    APP_CHANNEL: master
  static-analysis-bot:
    ANALYZERS:
      - clang-tidy
      - clang-format
      - infer
      - mozlint
    PHABRICATOR:
      url: 'https://phabricator-dev.allizom.org/api/'
      api_key: api-YOURTOKEN
    REPORTERS:
      - reporter: mail
        emails:
          - <EMAIL>
      - reporter: phabricator

Just replace ``LOGIN`` with your username (e.g. ``michel``); ``api-YOURTOKEN`` with a new Conduit API Token from `Phabricator-dev https://phabricator-dev.allizom.org/settings/`_ (hint: on Bugzilla-dev, you can just "Sign in with GitHub"); and ``<EMAIL>`` with your email address.


2. Taskcluster client
"""""""""""""""""""""

You need to create a `Taskcluster client`_ to run the static analysis task on your computer.

Use the form to create a new client in your own namespace (the ``ClientId`` should be pre-filled with ``mozilla-auth0/ad|Mozilla-LDAP|login/``, simply add an explicit suffix, like ``static-analysis-dev``)

Add an explicit description, you can leave the ``Expires`` setting into the far future.

Add the Taskcluster scope needed to read the secret previously mentioned: ``secrets:get:repo:github.com/mozilla-releng/services:branch:master`` (or ``secrets:get:garbage/michel/staticanalysis-bot-dev`` if you're using your own secret)

You may also want to add the scope needed to send you an email report: ``notify:email:<EMAIL>`` (if your email is ``<EMAIL>``)

To summarize, you need to setup your client (if your login is ``michel``), like this:

============= ====================================================================
Key           Value
============= ====================================================================
ClientId      ``mozilla-auth0/ad|Mozilla-LDAP|michel/static-analysis-dev``
Description   My own static analysis dev. client
Client Scopes ``secrets:get:repo:github.com/mozilla-releng/services:branch:master``
              ``notify:email:<EMAIL>``
============= ====================================================================


.. warning::

  Save the **access token** provided by Taskcluster after creating your client, it won't be displayed afterwards


3. Project shell
""""""""""""""""

Run the following (where ``XXX`` is the Taskcluster access token):

.. code-block:: shell

  ./please shell staticanalysis/bot \
    --taskcluster-client-id="mozilla-auth0/ad|Mozilla-LDAP|michel/static-analysis-dev" \
    --taskcluster-access-token=XXX

Once the initial build finishes, you should get a green Nix shell, running in ``/app/src/staticanalysis/bot``.


4. Setup a Phabricator test
"""""""""""""""""""""""""""


.. note::
  Make sure your Taskcluster secret has a ``phabricator`` reporter setup, as follows (with a valid Phabricator uri & token):

  .. code-block:: yaml

    staticanalysis/bot:
      ...
      PHABRICATOR:
        url: 'https://phabricator-dev.allizom.org/api/'
        api_key: api-XXXX
      REPORTERS:
        - reporter: phabricator


The bot needs an environment variable ``PHABRICATOR`` containing the PHID of the diff to be reviewed.

So you'll need to do the following in the nix shell:

.. code-block:: shell

{'fraction_non_alphanumeric': 0.17599914511647788, 'fraction_numerical': 0.0036332549690104724, 'mean_word_length': 4.114207650273224, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2752953', 'n_tokens_mistral': 2786, 'n_tokens_neox': 2654, 'n_words': 945}