<jupyter_start><jupyter_text>Advanced testingUp till now we have been testing functions where the output is entirely predictable. In these cases, a handful of tests is usually enough to provide confidence that the software is working as expected. In the real world, however, you might be developing a complex piece of sofware to implement an entirely new algorithm, or model. In certain cases it might not even be clear what the expected outcome is meant to be. Things can be particularly challenging when the software is involves a stochastic element.Let us consider a class to simulate the behaviour of a dice. One is provided in the `dice` package. Let's import it and see how it works.<jupyter_code>from dice import Dice
help(Dice)<jupyter_output><empty_output><jupyter_text>How could we test that the dice is fair?Well, first of all we could check that the value of a dice roll is in range.```python dice/test/test_dice.pydef test_valid_roll():    """ Test that a dice roll is valid. """     Intialise a standard, six-sided dice.    dice = Dice()     Roll the dice.    roll = dice.roll()     Check that the value is valid.    assert roll > 0 and roll < 7```<jupyter_code>!pytest dice/test/test_dice.py::test_valid_roll<jupyter_output><empty_output><jupyter_text>Great, that worked. Although, it could just be a fluke...In practice, we need to check that the assertions hold repeatedly.```python dice/test/test_dice.pydef test_always_valid_roll():    """ Test that a dice roll is "always" valid. """     Intialise a standard, six-sided dice.    dice = Dice()     Roll the dice lots of times.    for i in range(0, 10000):        roll = dice.roll()         Check that the value is valid.        assert roll > 0 and roll < 7```<jupyter_code>!pytest dice/test/test_dice.py::test_always_valid_roll<jupyter_output><empty_output><jupyter_text>Okay, that's better. Or is it...![xkcd: random](https://imgs.xkcd.com/comics/random_number.png)Not again!Perhaps we should test the average value. We know that this should equal the sum of the faces of the dice, divided by the number of sides, i.e. 3.5 for a six-sided dice.```python dice/test/test_dice.pydef test_average():    """ Test that the average dice roll is correct. """     Intialise a standard, six-sided dice.    dice = Dice()     Work out the expected average roll.    exp = sum(range(1, 7)) / 6     Calculate the sum of the dice rolls.    total = 0    for i in range(0, 100000):        total += dice.roll()     Check that the average matches the expected value.    average = total / rolls    assert average == pytest.approx(3.5, rel=1e-2)```<jupyter_code>!pytest dice/test/test_dice.py::test_average<jupyter_output><empty_output><jupyter_text>Good... Hang on, hold your horses!<jupyter_code>(1 + 3 + 4 + 6) / 4<jupyter_output><empty_output><jupyter_text>Dang! We need to test that the _distrubtion_ of outcomes is correct, i.e. that each of the six possible outcomes is equally likely.```python dice/test/test_dice.pydef test_fair():    """ Test that a dice is fair. """     Intialise a standard, six-sided dice.    dice = Dice()     Set the number of rolls.    rolls = 1000000     Create a dictionary to hold the tally for each outcome.    tally = {}    for i in range(1, 7):        tally[i] = 0     Roll the dice 'rolls' times.    for i in range(0, rolls):        tally[dice.roll()] += 1     Assert that the probability is correct.    for i in range(1, 7):        assert tally[i] / rolls == pytest.approx(1 / 6, 1e-2)```<jupyter_code>!pytest dice/test/test_dice.py::test_fair<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09629421673217294, 'fraction_numerical': 0.01403705783267827, 'mean_word_length': 3.8148648648648646, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '515783', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 1123, 'n_tokens_neox': 1065, 'n_words': 436, 'path': '05_advanced_testing.ipynb', 'repo_name': 'CDT-AIMLAC/testing_exercise', 'size': '3562'}