Adapted from Joseph Sheedy's material
Code which runs your application in as close to a real environment as feasible and validates its behavior
In dynamic interpreted languages like Python even show stopping syntax errors can hide until discovered in runtime, usually at the wrongtime.
#!/usr/bin/env python
def func():
jfdkls
while True:
print "> ",
if raw_input() == 'x':
func()
What should be tested?
The percentage of code which gets run in a test is known as the coverage.
100% coverage is an ideal to strive for. But the decision on when and what to test should take into account the volatility of the project. Tests require maintenance.
Unit testing tools
http://docs.python.org/2/library/unittest.html
create a new subclass of unittest.TestCase
name test methods test_foo so the test runner finds them
make calls to the self.assert* family of methods to validate results
import unittest
class TestTest(unittest.TestCase):
def setUp(self):
pass
def test_add(self):
self.assertEqual(2+2, 4)
def test_len(self):
self.assertEqual(len('foo'), 3)
TestCase contains a number of methods named assert* which can be used for validation, here are a few common ones:
See a full list at http://docs.python.org/2/library/unittest.html#assert-methods or dir(unittest.TestCase)
Test fixtures are a fixed baseline for tests to run from consistently, also known as test context
Fixtures can be set up fresh before each test, once before each test case, or before an entire test suite
unittest provides fixture support via these methods:
Why can't we just test if .5 == .5 ?
3*.15 == .45
Out[19]: False
In [24]: 3*.15 * 10 / 10 == .45
Out[24]: True
There are an infinite number of floating point numbers, so they are stored as an approximation in computing hardware.
Floating point numbers are stored in IEEE 754 64-bit double precision format, which allows 1 bit for the sign, 11 bits for the exponent, and the remaining 52 for the fraction
So we can count on 16 digits of precision in decimal:
In [39]: len(str(2**52))
Out[39]: 16
In [40]: .1+.2
Out[40]: 0.30000000000000004
In [41]: len('3000000000000000')
Out[41]: 16
# with repeated operations, the errors eventually build up: here's multiplying by '1' 10 billion times:
In [64]: x=1
In [69]: for i in xrange(10000000000): x *= (.1 + .2)/.3
Verifies that two floating point values are close enough to each other. Add a places keyword argument to specify the number of significant digits.
import unittest
class TestAlmostEqual(unittest.TestCase):
def setUp(self):
pass
def test_floating_point(self):
self.assertEqual(3*.15, .45)
def test_almost_equal(self):
self.assertAlmostEqual(3*.15, .45, places=7)
Call unittest.main() right in your module
if __name__ == "__main__":
unittest.main()
If it gets cumbersome with many TestCases, organize the tests into a test suite
Test suites group test cases into a single testable unit
import unittest
from calculator_test import TestCalculatorFunctions
suite = unittest.TestLoader().loadTestsFromTestCase(TestCalculatorFunctions)
unittest.TextTestRunner(verbosity=2).run(suite)
A test runner which autodiscovers test cases
Nose will find tests for you so you can focus on writing tests, not maintaining test suites
Any file matching the testMatch conditions* will be searched for tests. They can't be executable!
Running your tests is as easy as
$ nosetests
https://nose.readthedocs.org/en/latest/finding_tests.html
*defined as self.testMatch = re.compile(r'(?:^|[\\b_\\.%s-])[Tt]est' % os.sep)
Many plugins exist for nose, such as code coverage:
# requires full path to nosetests:
$ ~/virtualenvs/uwpce/bin/nosetests --with-coverage
or drop in to the debugger on failure
$ nosetests --pdb
or parallel process your tests. Remember, unit tests should be independent of each other:
$ nosetests --processes=5
To run coverage on your test suite:
coverage run my_program.py arg1 arg2
This generates a .coverage file. To analyze it on the console:
coverage report
Else generate an HTML report in the current directory:
coverage html
To find out coverage across the standard library, add -L:
-L, --pylib Measure coverage even inside the Python installed
library, which isn't done by default.
consider the following code:
x = False # 1
if x: # 2
print "in branch" # 3
print "out of branch" # 4
We want to make sure the branch is being bypassed correctly in the False case
Track which branch destinations were not visited with the --branch option to run
coverage run --branch myprog.py
Tests placed in docstrings to demonstrate usage of a component to a human in a machine testable way
def square(x):
"""Squares x.
>>> square(2)
4
>>> square(-2)
4
"""
return x * x
if __name__ == '__main__':
import doctest
doctest.testmod()
As of Python 2.6, the __main__ check is unnecessary:
python -m doctest -v example.py
Now generate documentation, using epydoc for example:
$ epydoc example.py
http://docs.python.org/2/library/doctest.html
http://www.python.org/dev/peps/pep-0257/
In TDD, the tests are written the meet the requirements before the code exists.
Once the collection of tests passes, the requirement is considered met.
We don't always want to run the entire test suite. In order to run a single test with nose:
nosetests calculator_test.py:TestCalculatorFunctions.test_add
--with-coverage --cover-branches
Introduced in Python 2.5
If you've been opening files using "with" (and you probably should be), you have been using context managers:
with open("file.txt", "w") as f:
f.write("foo")
A context manager is just a class with __enter__ and __exit__ methods defined to handle setting up and tearing down the context
Provides generalizable execution contexts in which setup and teardown of context are executed no matter what happens
This allows us to do things like setup/teardown and separate out exception handling code
Define __enter__(self) and __exit__(self, type, value, traceback) on a class
If __exit__ returns a true value, a caught exception is not re-raised
For example :
import os, random, shutil, time
class TemporaryDirectory(object):
"""A context manager for creating a temporary directory which gets destroyed on context exit"""
def __init__(self,directory):
self.base_directory = directory
def __enter__(self):
# set things up
self.directory = os.path.join(self.base_directory, str(random.random()))
os.makedirs(self.directory)
return self.directory
def __exit__(self, type, value, traceback):
# tear it down
shutil.rmtree(self.directory)
with TemporaryDirectory("/tmp/foo") as dir:
# write some temp data into dir
with open(os.path.join(dir, "foo.txt"), 'wb') as f:
f.write("foo")
time.sleep(5)
Create a context manager which prints information on all exceptions which occur in the context and continues execution
with YourExceptionHandler():
print "do some stuff here"
1/0
print "should still reach this point"
Why might using a context manager be better than implementing this with try..except..finally ?
Also see the contextlib module
Consider the application in the examples/wikidef directory. Give the command line utility a subject, and it will return a definition.
./define.py Robot | html2text
How can we test our application code without abusing (and waiting for) Wikipedia?
Mock objects replace real objects in your code at runtime during test
This allows you to test code which calls these objects without having their actual code run
Useful for testing objects which depend on unimplemented code, resources which are expensive, or resources which are unavailable during test execution
The MagickMock class will keep track of calls to it so we can verify that the class is being called correctly, without having to execute the code underneath
import mock
mock_object = mock.MagicMock()
mock_object.foo.return_value = "foo return"
print mock_object.foo.call_count
print mock_object.foo()
print mock_object.foo.call_count
# raise an exception by assigning to the side_effect attribute
mock_object.foo.side_effect = Exception
mock_object.foo()
patch acts as a function decorator, class decorator, or a context manager
Inside the body of the function or with statement, the target is patched with a new object. When the function/with statement exits the patch is undone
# patch with a decorator
@patch.object(Wikipedia, 'article')
def test_article_success_decorator_mocked(self, mock_method):
article = Definitions.article("Robot")
mock_method.assert_called_once_with("Robot")
# patch with a context manager
def test_article_success_context_manager_mocked(self):
with patch.object(Wikipedia, 'article') as mock_method:
article = Definitions.article("Robot")
mock_method.assert_called_once_with("Robot")
When define.py is given the name of a non-existant article, an exception is thrown.
Add a new test that confirms this behavior
/