<jupyter_start><jupyter_text>10) Writing unit tests and running from anywhere on your machine Related references:- https://docs.python-guide.org/writing/tests/- https://jeffknupp.com/blog/2013/12/09/improve-your-python-understanding-unit-testing/- https://docs.python.org/3/library/unittest.html We always want to test that our program works. Last time, we did that manually--I looked at the output and saw it was what I expected and wanted. We also talked about the need to automate tests, so we can be sure that, as we add more functionality, we don't accidentally break something. This will also allow us to check that our program behaves as desired on multiple architectures. Today, we'll discuss how to do so.Writing tests for Python is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. We try to have tests both for normal behavior and for error conditions. Tests live in the `tests` directory, where every file that includes tests has a `test_` prefix.  What is meant by unit testing?It is testing small units of code, e.g. individual functions, to help isolate the location of the error.In addition, I always test the "main" function to make sure individual units connect correctly.`unittest` is also a test module in the Python standard library. Creating test cases is accomplished by subclassing unittest.TestCase (see below)  Examples of types of errors to look for1. *Syntax errors*: often checks that the user typed in a valid argument, and gives a useful error message if such a mistake is caught.1. *Logical errors*: when the algorithm used is not correct (either originally, or error created upon later editing of the code)1. *Unexpected input/edge cases*: e.g. a program is only meant to work with positive values (e.g. a zero is entered and a log will be taken)What type of error does the test below check for?<jupyter_code># A very basic unittest

import unittest

def fun(x):
    return x + 1

class MyTest(unittest.TestCase):
    def test(self):
        self.assertEqual(fun(3), 4)<jupyter_output><empty_output><jupyter_text>The heart of the test is the assertion step at the end. There are many options, more even than those listed below. There are further notes on how all the following work at https://docs.python.org/3/library/unittest.html.| Method                     | Checks that                                                                   ||----------------------------|-------------------------------------------------------------------------------|| assertEqual(a, b)          | a == b                                                                        || assertNotEqual(a, b)       | a != b                                                                        || assertTrue(x)	             | bool(x) is True                                                               || assertFalse(x)             | bool(x) is False                                                              || assertIs(a, b)             | a is b                                                                        || assertIsNot(a, b)          | a is not b                                                                    || assertIsNone(x)            | x is None                                                                     || assertIsNotNone(x)         | x is not None                                                                 | | assertIn(a, b)             | a in b                                                                        | | assertNotIn(a, b)          | a not in b                                                                    | | assertAlmostEqual(a, b)    | round(a-b, 7) == 0                                                            | | assertNotAlmostEqual(a, b) | round(a-b, 7) != 0                                                            | | assertGreater(a, b)        | a > b                                                                         || assertGreaterEqual(a, b)   | a >= b                                                                        || assertLess(a, b)           | a < b                                                                         || assertLessEqual(a, b)      | a <= b                                                                        || assertCountEqual(a, b)	 | a and b have the same elements in the same number, regardless of their order  | Generally, the functions we are testing will be in a different file than the tests. I've set up tests for our sample arthritis project (https://github.com/team-mayes/arthritis_proj_demo). Let's walk through those tests to get a feel for the types of tests we want to make.First, note that I've reorganized my folders a little bit, separating out `tests` and making a subfolder with files for testing the script `data_proc.py`. I like to have a separate test for each script, and a separate folder for files for testing that script.![Screen shot](images/lect10_reorg.png) Now let's look at `test_data_proc.py`. As usual, the beginning includes imports of libraries. Not that we have to import the script that we want to test.<jupyter_code>"""
Unit and regression test for data_proc.py
"""
import errno
import os
import sys
import unittest
from contextlib import contextmanager
from io import StringIO
import numpy as np
import logging
from arthritis_proj.data_proc import main, data_analysis<jupyter_output><empty_output><jupyter_text>The `logging` library is helpful to log or perform certain actions only under certain conditions. You'll see how this is used in the tests below.<jupyter_code># logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
DISABLE_REMOVE = logger.isEnabledFor(logging.DEBUG)<jupyter_output><empty_output><jupyter_text>To refer to where particular files are, we want to use relative paths only, as we want these tests to work on many diffrent machines. What do you think the following lines do?<jupyter_code>CURRENT_DIR = os.path.dirname(__file__)
MAIN_DIR = os.path.join(CURRENT_DIR, '..')
TEST_DATA_DIR = os.path.join(CURRENT_DIR, 'data_proc')
PROJ_DIR = os.path.join(MAIN_DIR, 'arthritis_proj')
DATA_DIR = os.path.join(PROJ_DIR, 'data')
SAMPLE_DATA_FILE_LOC = os.path.join(DATA_DIR, 'sample_data.csv')

# Assumes running tests from the main directory
DEF_CSV_OUT = os.path.join(MAIN_DIR, 'sample_data_stats.csv')
DEF_PNG_OUT = os.path.join(MAIN_DIR, 'sample_data_stats.png')<jupyter_output><empty_output><jupyter_text>That does it for our preamble. Now I've defined a helper function that will be used by the tests below it. (FYI, as described on https://docs.python.org/3/library/errno.html, `errno.ENOENT` is a standard error message that means `No such file or directory`. Any questions on what this function does?<jupyter_code>def silent_remove(filename, disable=False):
    """
    Removes the target file name, catching and ignoring errors that indicate that the
    file does not exist.

    @param filename: The file to remove.
    @param disable: boolean to flag if want to disable removal
    """
    if not disable:
        try:
            os.remove(filename)
        except OSError as e:
            if e.errno != errno.ENOENT:
                raise<jupyter_output><empty_output><jupyter_text>Now let's get to the tests themselves, going through each line to see what they do:<jupyter_code>class TestMain(unittest.TestCase):
    # These tests make sure that the program can run properly from main
    def testSampleData(self):
        # Checks that runs with defaults and that files are created
        test_input = ["-c", DEFAULT_DATA_FILE_LOC]
        try:
            if logger.isEnabledFor(logging.DEBUG):
                main(test_input)
            # checks that the expected message is sent to standard out
            with capture_stdout(main, test_input) as output:
                self.assertTrue("sample_data_stats.csv" in output)

            self.assertTrue(os.path.isfile("sample_data_stats.csv"))
            self.assertTrue(os.path.isfile("sample_data_stats.png"))
        finally:
            silent_remove(DEF_CSV_OUT, disable=DISABLE_REMOVE)
            silent_remove(DEF_PNG_OUT, disable=DISABLE_REMOVE)
            
class TestMainFailWell(unittest.TestCase):
    def testMissingFile(self):
        test_input = ["-c", "ghost.txt"]
        if logger.isEnabledFor(logging.DEBUG):
            main(test_input)
        with capture_stderr(main, test_input) as output:
            self.assertTrue("ghost.txt" in output)<jupyter_output><empty_output><jupyter_text>`capture_stdout` and `capture_stderr` called functions at the bottom of the file. Don't worry about understanding each line of the bellow. The main point is that these allow any messages sent to standard out or standard error, respectively, to be captured as strings.<jupyter_code># Utility functions

# From http://schinckel.net/2013/04/15/capture-and-test-sys.stdout-sys.stderr-in-unittest.testcase/
@contextmanager
def capture_stdout(command, *args, **kwargs):
    # pycharm doesn't know six very well, so ignore the false warning
    # noinspection PyCallingNonCallable
    out, sys.stdout = sys.stdout, StringIO()
    command(*args, **kwargs)
    sys.stdout.seek(0)
    yield sys.stdout.read()
    sys.stdout = out


@contextmanager
def capture_stderr(command, *args, **kwargs):
    # pycharm doesn't know six very well, so ignore the false warning
    # noinspection PyCallingNonCallable
    err, sys.stderr = sys.stderr, StringIO()
    command(*args, **kwargs)
    sys.stderr.seek(0)
    yield sys.stderr.read()
    sys.stderr = err<jupyter_output><empty_output><jupyter_text>Separately, the tests below checked that the function `data_analysis()` behaves as expected.<jupyter_code>class TestDataAnalysis(unittest.TestCase):
    def testSampleData(self):
        # Tests that the np array generated by the data_analysis function matches saved expected results
        csv_data = np.loadtxt(fname=SAMPLE_DATA_FILE_LOC, delimiter=',')
        analysis_results = data_analysis(csv_data)
        expected_results = np.loadtxt(fname=os.path.join(TEST_DATA_DIR, "sample_data_results.csv"), delimiter=',')
        self.assertTrue(np.allclose(expected_results, analysis_results))

    def testSampleData2(self):
        # A second check, with slightly different values, of the data_analysis function
        csv_data = np.loadtxt(fname=os.path.join(TEST_DATA_DIR, "sample_data2.csv"), delimiter=',')
        analysis_results = data_analysis(csv_data)
        expected_results = np.loadtxt(fname=os.path.join(TEST_DATA_DIR, "sample_data2_results.csv"), delimiter=',')
        self.assertTrue(np.allclose(expected_results, analysis_results))<jupyter_output><empty_output><jupyter_text>Let's add another test together. I want to see what happens if I have different number of days of data for different patients. I started with a new file () in which I deleted some days of data from patients 2 and three (want to see what is different? Do a file compare!).![Screen shot](images/lect10_missing-cols.png) Now let's see what happens when we run this, which I will do from the beginnings of a new test. Note that I moved a test into a new class. I do this to help people who might use my tests to figure out how to run the program.<jupyter_code>class TestMainBadInput(unittest.TestCase):
    def testMissingFile(self):
        test_input = ["-c", "ghost.txt"]
        main(test_input)
        with capture_stderr(main, test_input) as output:
            self.assertTrue("ghost.txt" in output)

    def testDataDiffNumCols(self):
        input_file = os.path.join(TEST_DATA_DIR, "sample_data3_diff_cols.csv")
        test_input = ["-c", input_file]
        main(test_input)<jupyter_output><empty_output><jupyter_text>Running `testDataDiffNumCols`, I get the following error: ![Screen shot](images/lect10_error.png) I'll take this information, specifically the type of error that I'm getting, and do a try-catch loop for this error. I know that this problem happens in line 67 of data_proc.py, so I'll go there (by clicking on the link) to see where I can catch this exception. ![Screen shot](images/lect10_error-loc.png) We are already in a try-catch loop, but it only caught an `IOError`. We now can add the so I'll add the `ValueError`.<jupyter_code>def parse_cmdline(argv):
    """
    Returns the parsed argument list and return code.
    `argv` is a list of arguments, or `None` for ``sys.argv[1:]``.
    """
    if argv is None:
        argv = sys.argv[1:]

    # initialize the parser object:
    parser = argparse.ArgumentParser()
    parser.add_argument("-c", "--csv_data_file", help="The location (directory and file name) of the csv file with "
                                                      "data to analyze",
                        default=DEFAULT_DATA_FILE_LOC)
    args = None
    try:
        args = parser.parse_args(argv)
        args.csv_data = np.loadtxt(fname=args.csv_data_file, delimiter=',')
    except IOError as e:
        warning("Problems reading file:", e)
        parser.print_help()
        return args, IO_ERROR
    except ValueError as e:
        warning("Read invalid data:", e)
        parser.print_help()
        return args, INVALID_DATA
    
    return args, SUCCESS<jupyter_output><empty_output><jupyter_text>Let's try running our test again.![Screen shot](images/lect10_no-asserts.png) Now we get the behavior we want, but we are not yet actually testing anything with an assert statement. Let's do that by checking that we get the error we expect. *FYI*: I added a clause that the program will be run on its own only if the debugger is on, but we are not getting anything from running it without capturing an error message that we will use for testing. It can be nice to see the whole output message, which we would get if we have make the debugging mode active.<jupyter_code>class TestMainFailWell(unittest.TestCase):
    def testMissingFile(self):
        test_input = ["-c", "ghost.txt"]
        if logger.isEnabledFor(logging.DEBUG):
            main(test_input)
        with capture_stderr(main, test_input) as output:
            self.assertTrue("ghost.txt" in output)

    def testDataDiffNumCols(self):
        input_file = os.path.join(TEST_DATA_DIR, "sample_data3_diff_cols.csv")
        test_input = ["-c", input_file]
        if logger.isEnabledFor(logging.DEBUG):
            main(test_input)
        with capture_stderr(main, test_input) as output:
            self.assertTrue("Wrong number of columns" in output)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08670916443896086, 'fraction_numerical': 0.00363287408321338, 'mean_word_length': 2.5611422992433486, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 56, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '248584', 'license': 'MIT', 'n_tokens_mistral': 3871, 'n_tokens_neox': 3768, 'n_words': 1445, 'path': 'source/notebooks/lecture10_unit_testing.ipynb', 'repo_name': 'samueldy/mecheng-599-006', 'size': '14589'}