<commit_before>Testing SecureDrop
==================

The SecureDrop project ships both application code for running on servers
hosted on-site at news organizations, as well as configuration scripts
for provisioning the servers to accept updates to the application code,
and to harden the system state. Therefore testing for the project includes
**Application Tests** for validating that the app code behaves as expected,
and **Configuration Tests** to ensure that the servers are appropriately
locked down, and able to accept updates to the app code.

Application Tests
-------------------

The application test suite uses:

  * Pytest_
  * Selenium_
  * Coveralls_

The functional tests use an outdated version of Firefox chosen specifically
for compatibility with Selenium, and a rough approximation of the most recent
Tor Browser.

.. note:: We're working on running the Selenium tests in Tor Browser.
          See `GitHub #1629`_ for more info.

.. _`GitHub #1629`: https://github.com/freedomofpress/securedrop/pull/1629

.. todo:: Flesh out explanation of the application tests. Decide what's
          important for devs to know; at a high-level, organization could be:

            * Installation
            * Running the application tests
            * any relevant info on editing them

          which matches the layout used for the Config tests.

.. _Pytest: https://docs.pytest.org/en/latest/
.. _Selenium: http://docs.seleniumhq.org/docs/
.. _Coveralls: https://github.com/coveralls-clients/coveralls-python

Installation
^^^^^^^^^^^^

The application tests are installed automatically in the Development
and App-Staging VMs, based on the contents of
``securedrop/requirements/test-requirements.txt``.
If you wish to change the dependencies, see :ref:`updating_pip_dependencies`.


.. _config_tests:

Configuration Tests
-------------------

Testinfra_ tests verify the end state of the Vagrant machines. Any
changes to the Ansible configuration should have a corresponding
spectest.

.. _Testinfra: https://testinfra.readthedocs.io/en/latest/

Installation
^^^^^^^^^^^^

.. code:: sh

    pip install -r securedrop/requirements/develop-requirements.txt

Running the config tests
^^^^^^^^^^^^^^^^^^^^^^^^

In order to run the tests, first create and provision the VM you intend
to test:

.. code:: sh
<commit_msg>Adds docs on running the app tests<commit_after>Testing SecureDrop
==================

The SecureDrop project ships both application code for running on servers
hosted on-site at news organizations, as well as configuration scripts
for provisioning the servers to accept updates to the application code,
and to harden the system state. Therefore testing for the project includes
**Application Tests** for validating that the app code behaves as expected,
and **Configuration Tests** to ensure that the servers are appropriately
locked down, and able to accept updates to the app code.

Application Tests
-------------------

The application test suite uses:

  * Pytest_
  * Selenium_
  * Coveralls_

The functional tests use an outdated version of Firefox chosen specifically
for compatibility with Selenium, and a rough approximation of the most recent
Tor Browser.

.. note:: We're working on running the Selenium tests in Tor Browser.
          See `GitHub #1629`_ for more info.

.. _`GitHub #1629`: https://github.com/freedomofpress/securedrop/pull/1629

.. todo:: Flesh out explanation of the application tests. Decide what's
          important for devs to know; at a high-level, organization could be:

            * Installation
            * Running the application tests
            * any relevant info on editing them

          which matches the layout used for the Config tests.

.. _Pytest: https://docs.pytest.org/en/latest/
.. _Selenium: http://docs.seleniumhq.org/docs/
.. _Coveralls: https://github.com/coveralls-clients/coveralls-python

Installation
^^^^^^^^^^^^

The application tests are installed automatically in the Development
and App-Staging VMs, based on the contents of
``securedrop/requirements/test-requirements.txt``.
If you wish to change the dependencies, see :ref:`updating_pip_dependencies`.

Running the application tests
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tests can be run inside the development VM:

.. code:: sh

    vagrant ssh development
    cd /vagrant/securedrop
    pytest -v tests

Or the app-staging VM:

.. code:: sh

    vagrant ssh app-staging
    sudo su www-data -s /bin/bash
    cd /var/www/securedrop
    pytest -v tests

For explanation of the difference between these machines, see
:doc:`virtual_environments`.

.. _config_tests:

Configuration Tests
-------------------

Testinfra_ tests verify the end state of the Vagrant machines. Any
changes to the Ansible configuration should have a corresponding
spectest.

.. _Testinfra: https://testinfra.readthedocs.io/en/latest/

Installation
^^^^^^^^^^^^

.. code:: sh

    pip install -r securedrop/requirements/develop-requirements.txt

Running the config tests
^^^^^^^^^^^^^^^^^^^^^^^^

In order to run the tests, first create and provision the VM you intend
to test:

.. code:: sh

{'fraction_non_alphanumeric': 0.107421875, 'fraction_numerical': 0.0046875, 'mean_word_length': 3.592825112107623, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5511901', 'n_tokens_mistral': 1394, 'n_tokens_neox': 1317, 'n_words': 584}