<commit_before>==============================
 addons.mozilla.org Validator
==============================

The AMO Validator is a tool designed to scan Mozilla add-on packages for
problems such as security vulnerabilities, exploits, spamware and badware,
and lots of other gunk. By using a combination of various techniques and
detection mechanisms, the validator is capable of being both efficient as well
as thorough.

-------
 Setup
-------

Prerequisites
=============

Python Libraries:

- argparse
- cssutils
- rdflib
- fastchardet

Python Libraries for Testing:

- nose
- coverage

You can install everything you need for running and testing with ::

    pip install -r requirements.txt


Submodules
==========

The validator may require some submodules to work. Make sure to run ::

    git clone --recursive git://github.com/mozilla/amo-validator.git

so that you get all of the goodies inside.


Spidermonkey
============

A working copy of Spidermonkey (debug or non-debug is fine) is required. The
version installed must include support for the Parser API. At the time of this
writing the version of Spidermonkey included with major package managers does
not yet include the Parser API.

How do you know if your js binary has the Parser API? Run this::

    js -e 'Reflect;'

There should be no error output.

The best way to make sure you install the right Spidermonkey is to `clone`_ the
mozilla-central repo or `download the tip`_ (which is faster). Then build it
from source like this::

    cd mozilla-central
    cd js/src
    autoconf2.13
    ./configure
    make
    sudo cp dist/bin/js /usr/local/bin/js

You must use autoconf at *exactly* 2.13 or else it won't work. If you're using
`brew`_ on Mac OS X you can get autoconf2.13 with this::

    brew install https://gist.github.com/raw/765545/c87a75f2cf9e26c153970522e227f1c1cf63fb81/autoconf213.rb

If you don't want to put the ``js`` executable in your ``$PATH`` or you want it
in a custom path, you can define it as ``$SPIDERMONKEY_INSTALLATION`` in
your environment.

.. _`brew`: http://mxcl.github.com/homebrew/
.. _`clone`: http://hg.mozilla.org/mozilla-central/
.. _`download the tip`: http://hg.mozilla.org/mozilla-central/archive/tip.tar.bz2

---------
 Running
---------

Run the validator as follows ::

    python addon-validator <path to xpi> [-t <expected type>] [-o <output type>] [-v] [--boring] [--selfhosted] [--determined]

The path to the XPI should point to an XPI file.

-t                  The type that you expect your add-on to be detected as. The
                    list of types is listed below.
-o                  The type of output to generate. Types are listed below.
-v                  Enable verbose mode. Extra information will be displayed in
                    verbose mode, namely notices (informational messages),
                    Jetpack information if available, extra error info (like
                    contexts, file data, etc.), and error descriptions. This
                    only applies to ``-o text``.
--selfhosted        Disables messages that are specific to add-ons hosted on
                    AMO.
--boring            Disables colorful shell output.
--determined        Continue validating the remaining tiers of an add-on if one
                    tier has failed. Certain high-tiered tests may
                    inadvertently fail when this option is enabled for badly
                    malformed add-ons.
--target-appversion     Accepts a JSON string containing an object whose keys
                    are GUIDs and values are lists of version strings. In the
                    targetApplication and compatibility tests, the add-on's
                    predefined ``<em:targetApplication>`` values will be
                    overridden if its GUIDs match thoes from the JSON. E.g.:
                    ``{"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}": "5.*"}``
--for-appversions   Accepts a JSON string containing an object whose keys are
                    GUIDs and values are lists of version strings. If this
                    list is specified, non-inlinecompatibility tests will only
                    be run if they specifically target the applications and
                    veresions in this parameter. E.g.:<commit_msg>Fix CLI arg docs<commit_after>==============================
 addons.mozilla.org Validator
==============================

The AMO Validator is a tool designed to scan Mozilla add-on packages for
problems such as security vulnerabilities, exploits, spamware and badware,
and lots of other gunk. By using a combination of various techniques and
detection mechanisms, the validator is capable of being both efficient as well
as thorough.

-------
 Setup
-------

Prerequisites
=============

Python Libraries:

- argparse
- cssutils
- rdflib
- fastchardet

Python Libraries for Testing:

- nose
- coverage

You can install everything you need for running and testing with ::

    pip install -r requirements.txt


Submodules
==========

The validator may require some submodules to work. Make sure to run ::

    git clone --recursive git://github.com/mozilla/amo-validator.git

so that you get all of the goodies inside.


Spidermonkey
============

A working copy of Spidermonkey (debug or non-debug is fine) is required. The
version installed must include support for the Parser API. At the time of this
writing the version of Spidermonkey included with major package managers does
not yet include the Parser API.

How do you know if your js binary has the Parser API? Run this::

    js -e 'Reflect;'

There should be no error output.

The best way to make sure you install the right Spidermonkey is to `clone`_ the
mozilla-central repo or `download the tip`_ (which is faster). Then build it
from source like this::

    cd mozilla-central
    cd js/src
    autoconf2.13
    ./configure
    make
    sudo cp dist/bin/js /usr/local/bin/js

You must use autoconf at *exactly* 2.13 or else it won't work. If you're using
`brew`_ on Mac OS X you can get autoconf2.13 with this::

    brew install https://gist.github.com/raw/765545/c87a75f2cf9e26c153970522e227f1c1cf63fb81/autoconf213.rb

If you don't want to put the ``js`` executable in your ``$PATH`` or you want it
in a custom path, you can define it as ``$SPIDERMONKEY_INSTALLATION`` in
your environment.

.. _`brew`: http://mxcl.github.com/homebrew/
.. _`clone`: http://hg.mozilla.org/mozilla-central/
.. _`download the tip`: http://hg.mozilla.org/mozilla-central/archive/tip.tar.bz2

---------
 Running
---------

Run the validator as follows ::

    python addon-validator <path to xpi> [-t <expected type>] [-o <output type>] [-v] [--boring] [--selfhosted] [--determined]

The path to the XPI should point to an XPI file.

-t                  The type that you expect your add-on to be detected as. The
                    list of types is listed below.
-o                  The type of output to generate. Types are listed below.
-v                  Enable verbose mode. Extra information will be displayed in
                    verbose mode, namely notices (informational messages),
                    Jetpack information if available, extra error info (like
                    contexts, file data, etc.), and error descriptions. This
                    only applies to ``-o text``.
--selfhosted        Disables messages that are specific to add-ons hosted on
                    AMO.
--boring            Disables colorful shell output.
--determined        Continue validating the remaining tiers of an add-on if one
                    tier has failed. Certain high-tiered tests may
                    inadvertently fail when this option is enabled for badly
                    malformed add-ons.
--target-maxversion     Accepts a JSON string containing an object whose keys
                    are GUIDs and values are version strings. This will
                    override the max version that the add-on supports for the
                    corresponding application GUID.
--target-minversion     Identical to ``--target-minversion``, except overrides
                    the min version instead of the max.
--for-appversions   Accepts a JSON string containing an object whose keys are
                    GUIDs and values are lists of version strings. If this
                    list is specified, non-inlinecompatibility tests will only
                    be run if they specifically target the applications and
                    veresions in this parameter. E.g.:
{'fraction_non_alphanumeric': 0.09121859591453393, 'fraction_numerical': 0.013148626438131016, 'mean_word_length': 2.4842535787321065, 'pattern_counts': {'":': 1, '<': 10, '<?xml version=': 0, '>': 10, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1289730', 'n_tokens_mistral': 2411, 'n_tokens_neox': 2190, 'n_words': 1029}