<jupyter_start><jupyter_text>Testing in Python- hide: true- toc: true - badges: true- comments: true- categories: [tools] - Pytest docs: https://docs.pytest.org/en/6.2.x/getting-started.html- Pytest features: https://realpython.com/pytest-python-testing/- Parametrised testing with different small dataframes: https://www.statestitle.com/resource/pytest-for-data-scientists/- Read dataframes for fixtures:https://datatest.readthedocs.io/en/latest/tutorial/testing-pandas.html- Stuff to test: https://www.kdnuggets.com/2020/08/unit-test-data-pipeline-thank-yourself-later.html- Useful resources: https://gist.github.com/htorrence/df9deb99334420a5031c1a4624e83829- htorrence repo: https://github.com/htorrence/pytest_examples, and useful examples: https://github.com/htorrence/pytest_examples/blob/master/tests/test_basic.pyTesting: - Unit testing: test small pieces of code- Integration testing: test how different parts of code base interact- End-to-end testing: test entire program  Temporary notes - To import functions from `src` module, need to invoke pytest as `python -m pytest`, which adds current directory to sys [path](https://docs.pytest.org/en/6.2.x/usage.htmlcalling-pytest-through-python-m-pytest). Need to understand what's happening here.  Basic test for return value<jupyter_code>def convert_to_int(s):
    return int(s.replace(',', ''))<jupyter_output><empty_output><jupyter_text>Very basic test<jupyter_code>def test_convert_to_int():
    assert convert_to_int('1,200') == 1200<jupyter_output><empty_output><jupyter_text>More transparent test with message, which will show when AssertionError is raised<jupyter_code>def test_convert_to_int():
    actual = convert_to_int('1,200')
    expected = 1200
    message = f"convert_to_int('1,200') returned {actual} instead of {expected}."
    assert actual == expected, message<jupyter_output><empty_output><jupyter_text>Careful with floats: because of this:<jupyter_code>.1 + .1 + .1 == .3<jupyter_output><empty_output><jupyter_text>Use this:<jupyter_code>.1 + .1 + .1 == pytest.approx(.3)<jupyter_output><empty_output><jupyter_text>Testing for exceptions Use context manager that silences expected error if raised within context and raises an assertion error if expected error isn't raised.<jupyter_code>with pytest.raises(ValueError):
    raise ValueError
with pytest.raises(ValueError):
    pass<jupyter_output><empty_output><jupyter_text>Basic example:<jupyter_code>def incrementer(x):
    if isinstance(x, int):
        return x + 1
    elif isinstance(x, str):
        raise TypeError('Please enter a number')

def test_valueerror_on_string():
    example_argument = 'hello'
    with pytest.raises(TypeError):
        incrementer(example_argument)
        
test_valueerror_on_string()<jupyter_output><empty_output><jupyter_text>Test for correct error message<jupyter_code>def test_valueerror_on_string():
    example_argument = 'hello'
    with pytest.raises(TypeError) as exception_info:
        incrementer(example_argument)
    assert exception_info.match('Please enter a number')
        
test_valueerror_on_string()<jupyter_output><empty_output><jupyter_text>What's a "well-tested" function? Argument types:- Bad arguments    - Examples: incomplete args, wrong dimensions, wrong type, etc.    - Return value: exception- Special arguments    - Examples: values triggering special logic, boundary value (value between bad and good arguments and before or after values that raise special logic)    - Return value: expected value- Normal arguments    - Examples: All other values, test 2 or 3    - Return value: expected value  Keeping tests organised Principles to follow:- Mirror structure of `src` directory in `tests` directory- Name test modules as `test_`- Within test module, collect all tests for a single function in a class named `TestNameOfFunction` (from DataCamp: 'Test classes are containers inside test modules. They help separate tests for different functions within the test module, and serve as a structuring tool in the pytest framework.')<jupyter_code># test class layout

class TestNameOfFunction(object):
    
    def test_first_thing(self):
        pass
    
    def test_second_thing(self):
        pass<jupyter_output><empty_output><jupyter_text>Marking tests as expected to fail Sometimes we might want to differentiate between failing code and tests that we know won't run yet or under certain conditions (e.g. we might follow TDD and haven't written a test yet, or we know a function only runs in Python 3). In this case, we can apply decorators to either functions or classes Expect to fail always (e.g. because not implemented yet)<jupyter_code>class TestNameOfFunction(object):
    
    @pytest.mark.xfail
    def test_first_thing(self):
        pass
    
    @pytest.mark.xfail(reason='Not implemented yet.')
    def test_first_thing(self):
        """With optional reason arg."""
        pass
    
# or

@pytest.mark.xfail(reason='Not implemented yet.')
class TestNameOfFunction(object):
    
    def test_first_thing(self):
        pass
    
    def test_first_thing(self):
        """With optional reason arg."""
        pass<jupyter_output><empty_output><jupyter_text>Expect to fail under certain conditions (e.g. certain Python versions, operating systems, etc.).<jupyter_code>import sys

class TestNameOfFunction(object):
    
    @pytest.mark.skipif(sys.version_info < (3, 0), reason='Requires Python 3')
    def test_first_thing(self):
        """Only runs in Python 3."""
        pass
    
# or

@pytest.mark.skipif(sys.version_info < (3, 0), reason='Requires Python 3')
class TestNameOfFunction(object):
    
    def test_first_thing(self):
        """Only runs in Python 3."""
        pass<jupyter_output><empty_output><jupyter_text>Running pytests - `pytest` runs all tests- `pytest -x` stops after first failure- `pytest ` runs all tests in test module- `pytest ::` runs all tests in test module with specified node id- `pytest ::::` runs test with specified node id- `pytest -k ` runds tests that fit pattern    - `pytest -k ` runs all tests in specified class    - `pytest -k ` runs all tests in specified class except for `test_second_thing`- `pytest -r` show reasons- `pytest -rs` show reasons for skipped tests- `pytest -rx` show reasons for xfailed tests- `pytest -rsx` show reasons for skipped and xfailed  CI - [Travis CI](https://www.travis-ci.com)- [Python stuff](https://docs.travis-ci.com/user/languages/python/)- [Using Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/use-conda-with-travis-ci.html)- [Travis CI build info](https://app.travis-ci.com/github/fabiangunzinger/entropy/jobs/545437220)  Fixtures<jupyter_code># create raw and clean data files in fixture
@pytest.fixture
def raw_and_clean_data():
    raw_path = 'raw.csv'
    clean_path = 'clean.csv'
    with open(raw_path, 'w') as f:
        f.write(
            '1000, 40\n'
            '2000, 50\n'
        )
    yield raw_path, clean_path
    
    #teardown code so we start with clean env in next test
    os.remove(raw_path)
    os.remove(clean_path)
    
# use fixture in test
def test_on_raw_data(raw_and_clean_data):
    raw_path, clean_path = raw_and_clean_data
    preprocess(raw_path, clean_path)<jupyter_output><empty_output><jupyter_text>Useful alternative using `tempdir()` and fixture chaining:<jupyter_code>@pytest.fixture
def raw_and_clean_data(tempdir):
    raw_path = tempdir.join('raw.csv')
    clean_path = tempdir.join('clean.csv')
    with open(raw_path, 'w') as f:
        f.write(
            '1000, 40\n'
            '2000, 50\n'
        )
    yield raw_path, clean_path
    
    # no teardown needed<jupyter_output><empty_output><jupyter_text>MockingTesting functions independently of dependencies  Testing models To test models, use toy datasets for which I know the correct results and perform sanity-checks using assertions I can know. Tests for training function<jupyter_code>from models import train_model

def test_on_linear_data():
    """Can easily predict results precisely."""
    test_arg = np.array([[1, 3],
                         [2, 5],
                         [3, 7]])
    expected_slope = 2
    expected_intercept = 1
    
    slope, intercept = train_model(test_arg)
    assert slope == pytest.approx(expected_slope)
    assert intercept = pytest.approx(expected_intercept)
    
    
def test_on_positively_correlated_data():
    """Cannot easily predict result precisely,
    but can still assert that slope is positive
    as a sanity-check.
    """
    test_arg = np.array([[1, 4],
                         [2, 3],
                         [4, 8],
                         [3, 7]])
    slope, intercept = train_model(test_arg)
    assert slope > 0<jupyter_output><empty_output><jupyter_text>Tests for final model<jupyter_code>def model_test(test_set, slope, intercept):
    """Assert that R^2 is between 0 and 1."""
    rsq = ...
    assert 0 <= rsq <= 1<jupyter_output><empty_output><jupyter_text>Testing plotsOverall approach:1. Create baseline plot using plotting function and store as PNG image2. Test plotting function and compare to baselineInstall `pytest-mpl`<jupyter_code>def plot_best_fit_line(slope, intercept, x_array, y_array, title):
    """Plotting function to be tested."""
    pass

@pytest.mark.mpl_image_compare
def test_plot_for_linear_data():
    """Testing function.
    Under the hood, creates baseline and comparisons.
    """
    slope = 2
    intercept = 1
    x_array = np.array([1, 2, 3])
    y_array = np.array([3, 5, 7])
    title = 'Test plot for linear data'
    return plot_best_fit_line(slope, intercept, x_array, y_array, title)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10108751941998964, 'fraction_numerical': 0.014293112377006732, 'mean_word_length': 3.9747552807831017, 'pattern_counts': {'":': 0, '<': 69, '<?xml version=': 0, '>': 66, 'https://': 13, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '109374', 'license': 'Apache-2.0', 'n_tokens_mistral': 3091, 'n_tokens_neox': 2994, 'n_words': 862, 'path': '_notebooks/0000-09-27-python-unit-testing.ipynb', 'repo_name': 'fabiangunzinger/blog', 'size': '9655'}