<commit_before>cache: working with cross-testrun state
=======================================

.. versionadded:: 2.8

.. warning::

  The functionality of this core plugin was previosuly distributed
  as a third party plugin named ``pytest-cache``.  The core plugin
  is compatible regarding command line options and API usage except that you
  can only store/receive data between test runs that is json-serializable.

Usage
---------

The plugin provides two command line options to rerun failures from the
last ``py.test`` invocation:

* ``--lf`` (last failures) - to only re-run the failures.
* ``--ff`` (failures first) - to run the failures first and then the rest of
  the tests.

For cleanup (usually not needed), a ``--cache-clear`` option allows to remove
all cross-session cache contents ahead of a test run.

Other plugins may access the `config.cache`_ object to set/get 
**json encodable** values between ``py.test`` invocations.


Rerunning only failures or failures first
-----------------------------------------------

First, let's create 50 test invocation of which only 2 fail::

    # content of test_50.py
    import pytest

    @pytest.mark.parametrize("i", range(50))
    def test_num(i):
        if i in (17, 25):
           pytest.fail("bad luck")

If you run this for the first time you will see two failures::

    $ py.test -q
    .................F.......F........................
    ================================= FAILURES =================================
    _______________________________ test_num[17] _______________________________
    
    i = 17
    
        @pytest.mark.parametrize("i", range(50))
        def test_num(i):
            if i in (17, 25):
    >          pytest.fail("bad luck")
    E          Failed: bad luck
    
    test_50.py:6: Failed
    _______________________________ test_num[25] _______________________________
    
    i = 25
    
        @pytest.mark.parametrize("i", range(50))
        def test_num(i):
            if i in (17, 25):
    >          pytest.fail("bad luck")
    E          Failed: bad luck
    
    test_50.py:6: Failed
    2 failed, 48 passed in 0.04 seconds

If you then run it with ``--lf`` you will run only the two failing test
from the last run::

    $ py.test --lf
    =========================== test session starts ============================
    platform linux2 -- Python 2.7.6, pytest-2.7.3.dev426+ng9a90aac.d20150916, py-1.4.30, pluggy-0.3.0
    run-last-failure: rerun last 2 failures
    rootdir: /tmp/doc-exec-9, inifile: 
    collected 50 items
    
    test_50.py FF
    
    ================================= FAILURES =================================
    _______________________________ test_num[17] _______________________________
    
    i = 17
    
        @pytest.mark.parametrize("i", range(50))
        def test_num(i):
            if i in (17, 25):
    >          pytest.fail("bad luck")
    E          Failed: bad luck
    
    test_50.py:6: Failed
    _______________________________ test_num[25] _______________________________
    
    i = 25
    
        @pytest.mark.parametrize("i", range(50))
        def test_num(i):
            if i in (17, 25):
    >          pytest.fail("bad luck")
    E          Failed: bad luck
    
    test_50.py:6: Failed
    ================= 2 failed, 48 deselected in 0.02 seconds ==================

The last line indicates that 48 tests have not been run.

If you run with the ``--ff`` option, all tests will be run but the first
failures will be executed first (as can be seen from the series of ``FF`` and
dots)::

    $ py.test --ff
    =========================== test session starts ============================
    platform linux2 -- Python 2.7.6, pytest-2.7.3.dev426+ng9a90aac.d20150916, py-1.4.30, pluggy-0.3.0
    run-last-failure: rerun last 2 failures first
    rootdir: /tmp/doc-exec-9, inifile: 
    collected 50 items<commit_msg>some more doc refinements<commit_after>cache: working with cross-testrun state
=======================================

.. versionadded:: 2.8

.. warning::

  The functionality of this core plugin was previosuly distributed
  as a third party plugin named ``pytest-cache``.  The core plugin
  is compatible regarding command line options and API usage except that you
  can only store/receive data between test runs that is json-serializable.

Usage
---------

The plugin provides two command line options to rerun failures from the
last ``py.test`` invocation:

* ``--lf`` (last failures) - to only re-run the failures.
* ``--ff`` (failures first) - to run the failures first and then the rest of
  the tests.

For cleanup (usually not needed), a ``--cache-clear`` option allows to remove
all cross-session cache contents ahead of a test run.

Other plugins may access the `config.cache`_ object to set/get 
**json encodable** values between ``py.test`` invocations.


Rerunning only failures or failures first
-----------------------------------------------

First, let's create 50 test invocation of which only 2 fail::

    # content of test_50.py
    import pytest

    @pytest.mark.parametrize("i", range(50))
    def test_num(i):
        if i in (17, 25):
           pytest.fail("bad luck")

If you run this for the first time you will see two failures::

    $ py.test -q
    .................F.......F........................
    ================================= FAILURES =================================
    _______________________________ test_num[17] _______________________________
    
    i = 17
    
        @pytest.mark.parametrize("i", range(50))
        def test_num(i):
            if i in (17, 25):
    >          pytest.fail("bad luck")
    E          Failed: bad luck
    
    test_50.py:6: Failed
    _______________________________ test_num[25] _______________________________
    
    i = 25
    
        @pytest.mark.parametrize("i", range(50))
        def test_num(i):
            if i in (17, 25):
    >          pytest.fail("bad luck")
    E          Failed: bad luck
    
    test_50.py:6: Failed
    2 failed, 48 passed in 0.04 seconds

If you then run it with ``--lf``::

    $ py.test --lf
    =========================== test session starts ============================
    platform linux2 -- Python 2.7.6, pytest-2.7.3.dev426+ng9a90aac.d20150916, py-1.4.30, pluggy-0.3.0
    run-last-failure: rerun last 2 failures
    rootdir: /tmp/doc-exec-9, inifile: 
    collected 50 items
    
    test_50.py FF
    
    ================================= FAILURES =================================
    _______________________________ test_num[17] _______________________________
    
    i = 17
    
        @pytest.mark.parametrize("i", range(50))
        def test_num(i):
            if i in (17, 25):
    >          pytest.fail("bad luck")
    E          Failed: bad luck
    
    test_50.py:6: Failed
    _______________________________ test_num[25] _______________________________
    
    i = 25
    
        @pytest.mark.parametrize("i", range(50))
        def test_num(i):
            if i in (17, 25):
    >          pytest.fail("bad luck")
    E          Failed: bad luck
    
    test_50.py:6: Failed
    ================= 2 failed, 48 deselected in 0.02 seconds ==================

You have run only the two failing test from the last run, while 48 tests have
not been run ("deselected").

Now, if you run with the ``--ff`` option, all tests will be run but the first
previous failures will be executed first (as can be seen from the series
of ``FF`` and dots)::

    $ py.test --ff
    =========================== test session starts ============================
    platform linux2 -- Python 2.7.6, pytest-2.7.3.dev426+ng9a90aac.d20150916, py-1.4.30, pluggy-0.3.0
    run-last-failure: rerun last 2 failures first
    rootdir: /tmp/doc-exec-9, inifile: 
    collected 50 items
{'fraction_non_alphanumeric': 0.19168266154830454, 'fraction_numerical': 0.037108125399872044, 'mean_word_length': 2.519135524538496, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2831603', 'n_tokens_mistral': 2434, 'n_tokens_neox': 2194, 'n_words': 757}