Table of Contents
What is Python code testing and why does it matter?
In the software development life cycle, testing your code is essential. So, choosing – and using – the right Python testing tools should also be an essential part of writing good quality code.
Writing testing code and running it in parallel is now considered a good practice (that is often skipped by beginners). By implementing them wisely, testing would help to define your code’s intent more precisely and have a more decoupled architecture. If you don’t check your Python code BEFORE it reaches your end users, you run the risk of losing their goodwill if the program has bugs in it or, even worse, totally destroying your app’s reputation so nobody will want to use it in the first place.
The testing phases of the software development life cycle assist businesses in identifying all bugs and errors in the software prior to the implementation phase. If software bugs are not resolved prior to deployment, they can adversely affect the client’s business.
Furthermore, attempting to resolve these issues at a later stage can result in substantial costs. The longer you delay the detection of these issues, the greater the cost you are likely to face.
The following are different types of tests you should have and are often represented as a pyramid.
The lower you go on the pyramid, the smaller the unit of code being tested, and the more tests you should have. For example, one UI (E2E) test phase might test the flow of creating, editing, and then saving a document. That flow is composed of many different functions, each one of them should have its own unit tests.
In this article, we will limit our scope to Python’s built-in unit testing framework.
If you are looking for articles about Python profiling tools, read it here:
Does Python have a built-in unit testing library?
The Python standard library provides unittest as a built-in unit testing framework. The unittest framework was inspired by JUnit in Java and has a similar flavor to major unit testing frameworks in other programming languages.
It allows for test automation, the sharing of test setup and shutdown code, the grouping of tests into collections, and the independence of the tests from the reporting framework.
The following are some unittest important concepts in an object-oriented way:
| test fixture | Represents the preparation needed to perform one or more tests, and any associated cleanup actions. This could include things like making temporary or proxy databases, directories, or starting a server process. |
| test case | The individual testing unit. It looks for a specific response to a specific set of inputs. unittest includes a base class called TestCase that can be used to create new test cases. |
| test suite | A collection of test cases, test suites, or both. It is used to group tests that should be run together. |
| test runner | A component that orchestrates test execution and reports the results to the user. To indicate the results of running the tests, the runner may use a graphical interface, a textual interface, or return a special value. |
How do I get the unittest library?
As unittest is a built-in Python unit testing library, no further installation is needed.
How to perform unit testing with the unittest Python testing tools?
The following is a short script to test three-string methods (source: docs.python.org):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
import unittest class TestStringMethods(unittest.TestCase): def test_upper(self): self.assertEqual('foo'.upper(), 'FOO') def test_isupper(self): self.assertTrue('FOO'.isupper()) self.assertFalse('Foo'.isupper()) def test_split(self): s = 'hello world' self.assertEqual(s.split(), ['hello', 'world']) # check that s.split fails when the separator is not a string with self.assertRaises(TypeError): s.split(2) if __name__ == '__main__': unittest.main() |
Creating test cases is accomplished by subclassing unittest.TestCase. The code above has three individual tests that are:
test_upper: To test if we were able to successfully change the string cases to upper case.test_isupper: To verify the conditions of the given strings if it is upper or not.test_split: To check for an expected result, whether we successfully split the given string input.
The naming convention for performing testing is to inform the test runner about which methods represent tests.
The heart of each test is a call to assertEqual(), which checks for an expected result; assertTrue() or assertFalse(), which verifies a condition; or assertRaises(), which verifies that a specific exception is raised. These methods are used in place of the assert statement to allow the test runner to compile all test results and generate a report.
Here is the output on PyScripter IDE:
Save the script above as 01_testThreeStringMethods.py, and let’s try to perform unit testing from the command prompt in the next section.
How to conduct Python unit testing at runtime?
The following is the command to run Python unittest:
|
1 |
python -m unittest 01_testThreeStringMethods.py |
Or you can specifically call your test class like this:
|
1 |
python -m unittest 01_testThreeStringMethods.TestStringMethods |
For more detailed results (higher verbosity), pass this -v flag:
|
1 |
python -m unittest -v 01_testThreeStringMethods.py |
Or you can also use this command:
|
1 |
python -m unittest -v 01_testThreeStringMethods.TestStringMethods |
What does a failed Python unit test look like?
Try the following code, to see how the failure in unittest would look like (source: realpython.com):
|
1 2 3 4 5 6 7 8 9 10 11 12 |
import unittest class TestSum(unittest.TestCase): def test_sum(self): self.assertEqual(sum([1, 2, 3]), 6, "Should be 6") def test_sum_tuple(self): self.assertEqual(sum((1, 2, 2)), 6, "Should be 6") if __name__ == '__main__': unittest.main() |
If you execute the above code, you’ll see one success (indicated with .) and one failure (indicated with F). Here is the output on PyScripter IDE:
How to do test skipping and expecting failures with unittest?
unittest allows you to skip individual test methods as well as entire classes of tests. Furthermore, it supports marking a test as an “expected failure,” which is a broken test that will fail but should not be counted as a failure on a TestResult.
Try the following code, to see what the output of the skipping test and expected failure in unittest would look like (the following code is modified from docs.python.org, to show you different test outputs in a series of tests):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
import unittest import tensorflow import sys class MyTestCase(unittest.TestCase): @unittest.skip("demonstrating skipping") def test_nothing(self): self.fail("shouldn't happen") @unittest.skipIf(tensorflow.__version__ == "3.0.0", "not supported in this library version") def test_format(self): # Tests that work for only a certain version of the library. pass @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows") def test_windows_support(self): # Windows specific testing code pass @unittest.expectedFailure def test_maybe_skipped(self): if not external_resource_available(): # Test code that depends on the external resource, # and expecting failure due to external resource not available pass |
This is the output of running the example above in verbose mode:
|
1 |
python -m unittest -v 03_skippingTestsAndExpectedFailures.py |
We got 2 ok, 1 skipping test, and 1 expected failure.
What are the command line options provided by unittest?
unittest supports these command-line options:
-h, --help
To access a list of all the command-line options. Here is the command to do it:
|
1 |
python -m unittest -h |
-b, --buffer
The standard output and standard error streams are buffered during the test run. Output during a passing test is discarded. Output is echoed normally on test fail or error and is added to the failure messages.
-c, --catch
During the test run, Control-C waits for the current test to finish and then reports all the results so far.
A second Control-C raises the normal KeyboardInterrupt exception
-f, --failfast
On the first error or failure, the test is terminated.
-k
Only test methods and classes that match the pattern or substring are executed. This option may be used multiple times, in which case all test cases that match any of the given patterns are included.
Patterns that contain a wildcard character (*) are matched against the test name using fnmatch.fnmatchcase(); otherwise simple case-sensitive substring matching is used.
Patterns are matched against the fully qualified test method name as imported by the test loader.
For example, -k foo matches foo_tests.SomeTest.test_something, bar_tests.SomeTest.test_foo, but not bar_tests.FooTest.test_something.
--locals
Show local variables in tracebacks.
Amazing isn’t it? Now you can easily test your Python code using unittest.
Congratulations, now you have learned about built-in Python testing tools! Now you can implement it to write more professional code that is tested well before delivered to users!
Click here to start using PyScripter, a free, feature-rich, and lightweight IDE for Python developers.
Download RAD Studio to build more powerful Python GUI Windows Apps 5x Faster with Less Code.
Check out Python4Delphi which easily allows you to build Python GUIs for Windows using Delphi.
Also, check out DelphiVCL which easily allows you to build GUIs for Windows using Python.
References & further readings
[1] Chng, Z. M. (2022).
A Gentle Introduction to Unit Testing in Python. Machine Learning Mastery. machinelearningmastery.com/a-gentle-introduction-to-unit-testing-in-python
[2] Shaw, A. (Retrieved in 2022).
Getting Started With Testing in Python. Real Python. realpython.com/python-testing
[3] Python Software Foundation. (2022).
unittest — Unit testing framework. Python 3.11.1 documentation. docs.python.org/3/library/unittest.html
