Testing your code is critical throughout the software development life cycle. If follows from this that selecting – and employing – the appropriate Python testing tools should be an essential part of writing high-quality code.
Writing code tests and running them in parallel with the development of the actual core program is now considered best practice. Unfortunately, this kind of testing mechanism is often skipped by beginners. Testing, when used wisely, can help you define your code’s intent more precisely and create a more decoupled architecture. If you don’t test your Python code BEFORE it reaches your end users, you risk losing their trust if the program contains bugs, or, even worse, completely destroys your app’s reputation so that no one wants to use it. In this highly connected world, reputation plays a big part in potential success.
For more introduction to testing with Python, please read our article about Python’s built-in unit testing framework here:
Table of Contents
A summary of Test Driven Development (TDD)
Testing is so important to good software development that it has its own software development methodology; Test Driven Development (TDD).
Robert C. Martin [2] proposed three TDD rules:
- Writing production code is not permitted unless it is necessary to fix a failing unit test.
- You are not permitted to write more of a unit test than is required to fail, and compilation failures are considered failures.
- You may not write any more production code than is required to pass the one failing unit test.
The central concept of TDD is that we base our software development on a set of unit tests that we have written at the outset of coding, making unit testing the heart of the TDD software development process. This ensures that you have a test for every component you develop.
TDD is also biased toward smaller tests, which means more specific tests that test fewer components at a time. This helps with error detection, and smaller tests are also easier to read and understand because there are fewer components at work in a single run.
It does not imply that you must use TDD in your projects. However, you may consider that as a method of developing your code and the tests at the same time.
What role does testing play in machine learning?
In Machine Learning testing, the programmer enters input and observes the machine’s behavior and logic.
As a result, the goal of testing machine learning is to ensure that the machine’s logic remains consistent. Even after multiple calls to the program, the logic should not change.
Machine Learning is tested by supplying new or previously unseen data from a sample/distribution. As a result, generalization refers to a model’s ability to adapt to unknown data/scenarios (drawn from the same distribution as the one used to create the model).
What’s different about testing machine learning systems?
Here are the schematic differences between testing software vs testing machine learning system:
Traditional software systems
In traditional software systems, humans write the logic that interacts with data to produce the desired behavior. Our software tests help ensure that the written logic corresponds to the expected behavior.
Machine Learning software systems
However, in machine learning systems, humans provide desired behavior as examples during training, and the model optimization process produces the system’s logic.
Traditional software testing workflow
Machine Learning system testing workflow
Machine learning systems are more difficult to test because the logic of the system is not explicitly written. However, automated testing remains an important tool for creating high-quality software systems. These tests can generate a behavioral report of trained models, which can be used as a systematic approach to error analysis.
What exactly is NumPy?
NumPy is a well-known Python library for scientific computing. NumPy stands for “Numerical Python.” NumPy is the foundational Python package for array computing.
The operations in NumPy are divided into three main groups: Fourier Transform and Shape Manipulation, Mathematical and Logical Operations, and Linear Algebra and Random Number Generation. NumPy is written in C and Python to make it as fast as possible.
In addition to its obvious scientific applications, NumPy can be used as an efficient multi-dimensional container of generic data. It is possible to define arbitrary data types. This enables NumPy to easily and quickly integrate with a wide range of databases.
How do I get the NumPy library?
You can get NumPy using the pip command to your command prompt:
|
1 |
pip install numpy |
Or, if you use Anaconda Environment, you can install NumPy with:
|
1 |
conda install numpy |
NumPy testing guidelines
Run the tests from inside the Python script or IDE
NumPy can be tested in several ways. It can, for example, be run from within Python or an IDE:
|
1 2 3 |
import numpy numpy.test(label='slow') |
Run testing from the command prompt
Requirements
The following are the required libraries to perform NumPy testing in your command prompt:
pytest
You can get pytest using the pip command to your command prompt:
|
1 |
pip install pytest |
Or, if you use Anaconda Environment, you can install NumPy with conda install:
|
1 |
conda install -c anaconda pytest |
hypothesis
Install with pip:
|
1 |
pip install hypothesis |
Install with conda:
|
1 |
conda install -c conda-forge hypothesis |
Here is a basic example of Python testing using NumPy
After installing NumPy, pytest, and hypothesis properly, tests can then be run easily with the following command:
|
1 |
python -c "import numpy; numpy.test()" |
Excerpt of the output:
numpy.testing examples
In this section, we will show you several examples of numpy.testing, or NumPy’s test support.
numpy.testing.assert_allclose
Description
If two objects are not equal within the specified tolerance, an AssertionError is raised.
allclose(actual, desired, rtol, atol) is the equivalent test (note that allclose has different default values). The difference between actual and desired is compared to atol + rtol * abs (desired).
Parameters
actual: array_like. The array was obtained.
desired: array_like. Array desired.
rtol: float, optional. Relative tolerance.
atol: float, optional. Absolute tolerance.
equal_nan: bool, optional. NaNs will compare equally if True.
err_msg: str, optional. The error message is to be printed in case of failure.
verbose: bool, optional. If True, the values that conflict are appended to the error message.
Raises
AssertionError. If the actual and desired values do not match the specified precision.
No output is expected if the test passes.
An example of using PyScripter for Python unit tests with NumPy
Code example:
|
1 2 3 4 5 |
import numpy as np x = [1e-5, 1e-3, 1e-1] y = np.arccos(np.cos(x)) np.testing.assert_allclose(x, y, rtol=1e-5, atol=0) |
I set the x to not equal the rtol value to get the failed output:
|
1 2 3 4 5 |
import numpy as np x = [1e-7, 1e-3, 1e-1] y = np.arccos(np.cos(x)) np.testing.assert_allclose(x, y, rtol=1e-5, atol=0) |
Example of failed output in PyScripter IDE:
numpy.testing. assert_array_almost_equal_nulp
Description
Compare two arrays relatively to their spacing.
This is a relatively robust method for comparing two arrays with variable amplitudes.
Parameters
x, y: array_like. Input arrays.
nulp: int, optional. The maximum number of units in the last place for tolerance (see Notes in Reference [5]). The default value is 1.
Returns
None
Raises
AssertionError. If the spacing between x and y for one or more elements is larger than nulp.
Here’s another Python testing example
Code example:
|
1 2 3 4 5 |
import numpy as np x = np.array([1., 1e-10, 1e-20]) eps = np.finfo(x.dtype).eps np.testing.assert_array_almost_equal_nulp(x, x*eps/2 + x) |
Example of failed output in PyScripter IDE:
|
1 2 3 4 5 |
import numpy as np x = np.array([1., 1e-10, 1e-20]) eps = np.finfo(x.dtype).eps np.testing.assert_array_almost_equal_nulp(x, x*eps + x) |
numpy.testing.run_module_suite
Description
Run a test module.
Equivalent to calling $ nosetests <argv> <file_to_run> from the command line
Parameters
file_to_run: str, optional. The path to the test module, or None. Run the module from which this function is called by default.
argv: list of strings. Arguments are to be passed to the nose test runner. argv[0] is ignored. All command line arguments accepted by nosetests will work. If the value is None, sys.argv is used.
Here is an example of Python testing using PyScripter
Code example (example of the failed output in PyScripter IDE):
|
1 2 3 |
import numpy as np np.testing.run_module_suite(file_to_run="numpy/tests/test_numpy_version.py") |
The code above will run all the test routines in ‘test_numpy_version.py’:
Click here to start using PyScripter, a free, feature-rich, and lightweight IDE for Python developers.
Download RAD Studio to build more powerful Python GUI Windows Apps 5x Faster with Less Code.
Check out Python4Delphi which easily allows you to build Python GUIs for Windows using Delphi.
Also, check out DelphiVCL which easily allows you to build GUIs for Windows using Python.
References & further readings
[1] NumPy Developers. (2008-2022).
Test Support (numpy.testing). NumPy documentation. numpy.org/doc/stable/reference/routines.testing. html
[2] Martin, R. C. (2009).
The three laws of TDD. Robert C. Martin “Uncle Bob” personal website. butunclebob.com/ArticleS.UncleBob. TheThreeRulesOfTdd
[3] Jordan, J. (2020).
Effective testing for machine learning systems. Jeremy Jordan personal website. jeremyjordan.me/testing-ml
[4] NumPy Developers. (2008-2022).
Testing Guidelines. NumPy documentation. numpy.org/doc/stable/reference/testing.html
[5] NumPy Developers. (2008-2022).
numpy.testing.assert_array_almost_equal_nulp. NumPy documentation. numpy.org/doc/stable/reference/generated/ numpy.testing.assert_array_almost_equal_nulp.html
