Writing tests for code

  • Establish context
  • Factor our common pre- or post-work
  • Isolation, even with failures
  • "Fixtures"
  • Use "mocking" to abstract away dependencies
    • E.g. "fake" a module's input
    • Also a good "smell" for bad code that is not modular enough
    • Python has a mock library to create mock objects that track what happens to them
  • Testing Your Code (Hitchhiker's Guide to Python)

Styles of testing

Use testing libraries

Use dedicated testing functions because they typically handle failure better. E.g. when you use assert x == y in Python a test will just fail without informing you about the values of x and y. If you use the unittest module with self.assertEqual(x, y) you can an informative message instead, for free!

Testing libraries also support pre- and post operations like setUp and tearDown.

You can refactor tests too

Also apply common refactor practices to tests! If you keep writing the test self.assertEqual(p.get_value(), value) you can refactor this into a base class:

class BaseTestCase(unittest.TestCase):
"""
Base class for all tests object p
"""

    def assertValueEqual(self, p, value):
        self.assertEqual(p.get_value(), value)

The testing class for objects p now can inherit this base class.

You have the benefit that assertValueEqual is descriptive (but in this generic example obviously not).

Testing code that raises errors

If you want to test code that raises errors, assert a specific error is raised using code like:

with self.assertRaises(TypeError):
    # code that raises error e.g. for bad input goes here

This way the function fails (as expected) without throwing up the exception and failing the testing code.

Fixtures

Use fixtures when you want to ...

  • write a parameterized test i.e. pass arguments into the test method itself
    • And you might want that if the tested method requires that parameter.
  • reuse a block of code in multiple tests

E.g. see the pytest @pytest.fixture decorator. Decorate a function with this fixture. You can specify it's scope (e.g. on the module level). Pytest functions can get the fixture as an argument. The testing function will then automatically call the fixture if you use the fixture as an argument in the function you want to test.

Examples:

@pytest.fixture(scope="module")
def path():
    return "./data/bank_data.csv"

Or parameterized:

@pytest.fixture(scope="module", params=["./data/bank_data.csv", "./data/hospital_data.csv"])
def path(request):
    value = request.param
    return value

This fixture will execute for each parameter! This way you can test your function with multiple parameters with just a single call of the fixture!

Example usage:

def test_import_data(path):
    try:
        df = import_data(path)
    except FileNotFoundError as err:
        logging.error("File not found")
        raise err

See How to use fixtures in the pytest docs.

Passing values across test functions

If you have a sequence of functions that depend on each other, you need to store the intermediate test case results somehow.

TODO Is this integration testing?

In pytest you have two options:

  1. Store variables in the Pytest namespace
    • This is done using the conftest.py file
  2. Caching intermediate results

Pytest namespace in conftest.py

The fixtures defined in the root conftest.py are available in the whole test suite. The conftest.py file has a directory scope, so you can also define fixtures needed in subfolders only.

Example of basic conftest.py:

import pytest
def df_plugin():
    return None

def pytest_configure():
    pytest.df = df_plugin()

This defines df in the namespace of pytest so you can use it in any pytest tests. If you have a function read_data(), you can let the test for this function store the result using pytest.df = df. Other tests can then use pytest.df, which now refers to a dataframe.

Reads:

Caching values between tests

In Pytest, use request.config.cache.set() and request.config.cache.get(). Then, instead of using the Pytest namespace, the test_read_data test function would call request.config.cache.set('cached_df', df), which you can retrieve using the get version of the same function. N.B. request needs to be passed as an argument to the test function.

Docs

Q: What's the advantage of caching over the Pytest namespace?

Q: What's the advantage of using the Pytest namespace over caching??

Input parameters for tests

In some particular cases it may be nice to allow parameters to be passed into the testing suite. This is unlikely for unit tests, where you want to have fixed inputs with expected outputs, but may be suitable for data validation tests (see testing in data science). This way you can easily pass different data sets to a data validation test suite.

In pytest this can be achieved with a special fixture called parser that makes stores the parameter in the pytest namespace. You can add the following to conftest.py:

def pytest_add_option(parser):
    parser.addoption("--input_data", action="store")

When running pytest you can then do pytest . --input_data data/data.csv Because the result is stored in the pytest namespace, you can use the passed parameters in tests by calling request.config.option.input_data.