CodeLearn PythonPythonWindows

How To Add Python Testing Tools Into Machine Learning Code: Testing NumPy

How To Add Python Testing Tools Into Machine Learning Code Testing NumPy

Testing your code is critical throughout the software development life cycle. If follows from this that selecting – and employing – the appropriate Python testing tools should be an essential part of writing high-quality code.

Writing code tests and running them in parallel with the development of the actual core program is now considered best practice. Unfortunately, this kind of testing mechanism is often skipped by beginners. Testing, when used wisely, can help you define your code’s intent more precisely and create a more decoupled architecture. If you don’t test your Python code BEFORE it reaches your end users, you risk losing their trust if the program contains bugs, or, even worse, completely destroys your app’s reputation so that no one wants to use it. In this highly connected world, reputation plays a big part in potential success.

For more introduction to testing with Python, please read our article about Python’s built-in unit testing framework here:


A summary of Test Driven Development (TDD)

Testing is so important to good software development that it has its own software development methodology; Test Driven Development (TDD).

Robert C. Martin [2] proposed three TDD rules:

  1. Writing production code is not permitted unless it is necessary to fix a failing unit test.
  2. You are not permitted to write more of a unit test than is required to fail, and compilation failures are considered failures.
  3. You may not write any more production code than is required to pass the one failing unit test.

The central concept of TDD is that we base our software development on a set of unit tests that we have written at the outset of coding, making unit testing the heart of the TDD software development process. This ensures that you have a test for every component you develop.

TDD is also biased toward smaller tests, which means more specific tests that test fewer components at a time. This helps with error detection, and smaller tests are also easier to read and understand because there are fewer components at work in a single run.

It does not imply that you must use TDD in your projects. However, you may consider that as a method of developing your code and the tests at the same time.


What role does testing play in machine learning?

In Machine Learning testing, the programmer enters input and observes the machine’s behavior and logic.

As a result, the goal of testing machine learning is to ensure that the machine’s logic remains consistent. Even after multiple calls to the program, the logic should not change.

Machine Learning is tested by supplying new or previously unseen data from a sample/distribution. As a result, generalization refers to a model’s ability to adapt to unknown data/scenarios (drawn from the same distribution as the one used to create the model).


What’s different about testing machine learning systems?

Here are the schematic differences between testing software vs testing machine learning system: 

Traditional software systems

testing machine learning code python 1
The scheme of traditional software systems Source Reference 3

In traditional software systems, humans write the logic that interacts with data to produce the desired behavior. Our software tests help ensure that the written logic corresponds to the expected behavior.

Machine Learning software systems

testing machine learning code python 2
The scheme of Machine Learning software systems Source Reference 3

However, in machine learning systems, humans provide desired behavior as examples during training, and the model optimization process produces the system’s logic.

Traditional software testing workflow

testing machine learning code python 3
A typical workflow for software development Source Reference 3

Machine Learning system testing workflow

testing machine learning code python 4
A typical workflow for machine learning model development Source Reference 3
testing machine learning code python 5
A proposed workflow for developing high quality models Source Reference 3

Machine learning systems are more difficult to test because the logic of the system is not explicitly written. However, automated testing remains an important tool for creating high-quality software systems. These tests can generate a behavioral report of trained models, which can be used as a systematic approach to error analysis.


What exactly is NumPy?

NumPy is a well-known Python library for scientific computing. NumPy stands for “Numerical Python.” NumPy is the foundational Python package for array computing.

The operations in NumPy are divided into three main groups: Fourier Transform and Shape Manipulation, Mathematical and Logical Operations, and Linear Algebra and Random Number Generation. NumPy is written in C and Python to make it as fast as possible.

In addition to its obvious scientific applications, NumPy can be used as an efficient multi-dimensional container of generic data. It is possible to define arbitrary data types. This enables NumPy to easily and quickly integrate with a wide range of databases.


How do I get the NumPy library?

You can get NumPy using the pip command to your command prompt:

Or, if you use Anaconda Environment, you can install NumPy with:


NumPy testing guidelines

Run the tests from inside the Python script or IDE

NumPy can be tested in several ways. It can, for example, be run from within Python or an IDE:

testing numpy python 1
The NumPy testing outputs inside PyScripter IDE

Run testing from the command prompt


The following are the required libraries to perform NumPy testing in your command prompt:


You can get pytest using the pip command to your command prompt:

Or, if you use Anaconda Environment, you can install NumPy with conda install:


Install with pip:

Install with conda:

Here is a basic example of Python testing using NumPy

After installing NumPy, pytest, and hypothesis properly, tests can then be run easily with the following command:

Excerpt of the output:

testing numpy python 2
testing numpy python 3


numpy.testing examples

In this section, we will show you several examples of numpy.testing, or NumPy’s test support.



If two objects are not equal within the specified tolerance, an AssertionError is raised.

allclose(actual, desired, rtol, atol) is the equivalent test (note that allclose has different default values). The difference between actual and desired is compared to atol + rtol * abs (desired).


actual: array_like. The array was obtained.

desired: array_like. Array desired.

rtol: float, optional. Relative tolerance.

atol: float, optional. Absolute tolerance.

equal_nan: bool, optional. NaNs will compare equally if True.

err_msg: str, optional. The error message is to be printed in case of failure.

verbose: bool, optional. If True, the values that conflict are appended to the error message.


AssertionError. If the actual and desired values do not match the specified precision.

No output is expected if the test passes.

An example of using PyScripter for Python unit tests with NumPy

Code example:

I set the x to not equal the rtol value to get the failed output:

Example of failed output in PyScripter IDE:

testing numpy python 4

numpy.testing. assert_array_almost_equal_nulp


Compare two arrays relatively to their spacing.

This is a relatively robust method for comparing two arrays with variable amplitudes.


x, y: array_like. Input arrays.

nulp: int, optional. The maximum number of units in the last place for tolerance (see Notes in Reference [5]). The default value is 1.




AssertionError. If the spacing between x and y for one or more elements is larger than nulp.

Here’s another Python testing example

Code example:

Example of failed output in PyScripter IDE:

testing numpy python 5



Run a test module.

Equivalent to calling $ nosetests <argv> <file_to_run> from the command line


file_to_run: str, optional. The path to the test module, or None. Run the module from which this function is called by default.

argv: list of strings. Arguments are to be passed to the nose test runner. argv[0] is ignored. All command line arguments accepted by nosetests will work. If the value is None, sys.argv is used.

Here is an example of Python testing using PyScripter

Code example (example of the failed output in PyScripter IDE):

The code above will run all the test routines in ‘’:

testing numpy python 6

Click here to start using PyScripter, a free, feature-rich, and lightweight IDE for Python developers.

Download RAD Studio to build more powerful Python GUI Windows Apps 5x Faster with Less Code.

Check out Python4Delphi which easily allows you to build Python GUIs for Windows using Delphi.

Also, check out DelphiVCL which easily allows you to build GUIs for Windows using Python.

References & further readings

[1] NumPy Developers. (2008-2022).

Test Support (numpy.testing). NumPy documentation. html

[2] Martin, R. C. (2009).

The three laws of TDD. Robert C. Martin “Uncle Bob” personal website. TheThreeRulesOfTdd

[3] Jordan, J. (2020).

Effective testing for machine learning systems. Jeremy Jordan personal website.

[4] NumPy Developers. (2008-2022).

Testing Guidelines. NumPy documentation.

[5] NumPy Developers. (2008-2022).

numpy.testing.assert_array_almost_equal_nulp. NumPy documentation. numpy.testing.assert_array_almost_equal_nulp.html

Related posts

Unlock the Power of Python for Deep Learning with Generative Adversarial Networks (GANs) - The Engine behind DALL-E

CodeIDELearn PythonPythonPython GUITkinter

How To Make More Than 20 ChatGPT Prompts Work With Python GUI Builders And Matplotlib Library?

CodeIDELearn PythonPythonPython GUITkinter

How To Make More Than 20 ChatGPT Prompts Work With Python GUI Builders And Pillow Library?

CodeDelphiDelphiFMXLearn PythonProjectsPythonPython GUI

How To Create A Weather App With The Python Delphi Ecosystem and Weatherstack API

Leave a Reply

Your email address will not be published. Required fields are marked *