| Author: | Andrew J Todd esq. <andy47@halfcooked.com> |
|---|---|
| Date: | July, 2013 |
Have you tried unit testing? Always meant to add tests to your project but didn't know where to start? This article will provide a gentle introduction to unit testing your module, package or entire project.
There are many, many articles, books, blog posts and tutorial videos available explaining the benefits of unit testing so I'm not going to repeat them in great detail here. I have included some links to some great introductory tutorials to unit testing at the bottom of the page.
The assumption in this article is that you have at least heard of unit testing and may have even tried to write tests for a project in the past. If you started and then stopped, or thought that it all looked a bit too hard given that you had already written your code and got it working then this article is for you.
For the purposes of this article a project is anything that involves some Python code; from a single function up to a complete deliverable application system. The same principles apply to testing a function as to testing a whole application system.
Just so that we are all clear I'll quickly review some basic concepts. Unit testing is any repeatable activity that checks that the individual units of code within a module or application work as expected. This usually takes the form of code that tests your code. We don't usually need to write code that tests the test code.
Key features of unit testing are that tests are isolated from each other, execution is automated and that tests exercising the same parts of your application are grouped into batches.
Tests are isolated from each other and the code that they are testing. This makes finding and fixing problems that cause them to fail much easier.
For convenience unit testing only works if it is automated. There will usually be more test code than application code and running tests manually time after time is never a good idea. Ideally your full test suite should run from a single command. Indeed this is the approach taken with most test libraries, either via a convenience function or by a module that discovers your tests and runs them.
Test are grouped into batches to enable the developer to run sub-sets of the full test suite as and when required. It also makes adding or changing existing tests easier as they are easy to find.
Apart from the obvious reason - "to know that my code works" - there are other benefits to adopting unit testing for your project.
Unit testing will help you think about "how" you have written some code. In addition to the "what" it should be doing it never hurts to look at your implementation choices and figure out if they are appropriate. Code that has been unit tested will start to look different. The main effect is that adding unit tests will make each unit (function/method) of your appication smaller. This makes the code easier to understand, test and therefore change. Smaller units of code are good. A great explanation of this is from MichaelT on StackOverflow
Testing code that does lots of things is difficult.
Debugging code that does lots of things is difficult. The solution to both of these problems is to write code that doesn't do lots of things. Write each function so that it does one thing and only one thing. This makes them easy to test with a unit test (one doesn't need umpteen dozen unit tests).
A co-worker of mine has the phrase used he uses when judging if a given method needs to be broken up into smaller ones:
If, when describing the activity of the code to another programmer you use the word 'and', the method needs to be split into at least one more part.
Other benefits of unit testing include finding problems early, facilitating change, simplifying integration and improving design. For details check out the unit testing article at Wikipedia
The Python standard library comes with the unittest module. This is based on junit from Java. If you're a Java programmer then unittest may be the place to start.
For anyone else a great alternative is py.test. Starting to test your project with py.test is very easy. This is because there is very little boiler plate code required in your tests. It may be quick to start testing with py.test but it is far from a basic library. It has a full set of tools and testing capabilities when you need them. The traceback reporting is fantastic, it has really effective test collection and execution, test skipping and the ability to write parameterised tests. For more information see the py.test web site.
The reasons why I choose and continue to use py.test are the simple test collection, the lack of boilerplate and the ability to define set up and tear down functions at test, class or module level. For example for this function:
def parse_connection(connection_string):
"""Work out the user name and password in connection_string
Connection string should be of the form 'database://username@password'
"""
if '@' in connection_string:
# Username is characters between '://' and '@'
slash_position = connection_string.find('://')
at_position = connection_string.find('@')
user_name = connection_string[slash_position+3:at_position]
password = connection_string[at_position+1:]
return user_name, password
Here is the simplest possible test that I can write with unittest:
from parse_conn import parse_connection
import unittest
class InvalidInputs(unittest.TestCase):
def testNoAt(self):
"""parse_connection should raise an exception for an invalid connection string"""
self.assertRaises(ValueError, parse_connection, 'invalid uri')
if __name__ == "__main__":
unittest.main()
Whilst the same test in py.test is more simple:
from parse_conn import parse_connection
import py.test
def test_not_at():
py.test.raises(ValueError, parse_connection, 'invalid uri')
You don't need to know what a unittest.TestCase is or that you need to invoke unittest.main() to run your tests. When you run py.test at the command line without any options it will look in the current directory for any file name starting with test and run any function or method in that file where the name starts with test. You can, of course, be more specific about which modules and even tests to run but this out of the box functionality is where py.test beats unittest.
In essence I prefer py.test to unittest because it requires me to do less work when I want to start testing my code. Although with unittest2 the convenience of py.test is coming to the standard library.
The other reason I like py.test is the ability to define set up and tear down methods (to establish a consistent test state) at the test function, class or module level. This is actually in breach of the isolation principle of unit testing because more than one test can share the same set up and could potentially interfere with one another. But used properly it is a great example of the practicality beats purity aspect of the Zen of Python. Where you have test funcitons that require large, complex or expensive set up - like creating a whole database or a large randomised test data set - then repeating your set up for each test can be very time consuming. Long running tests are not run very often. If it's possible to share state between test functions without inhibiting others then sharing complex set up and tear down operations can greatly reduce the time it takes to run a full test suite.
More importantly, how do you start unit testing? Many people when faced with an application, a collection of files or even just a single module struggle to start testing. As the proverb says - the journey of a thousand miles starts with a single step. But what should that step be?
I know in the past I have thought that building a whole test suite for an application is just too much work and put it off until 'later'. It might take several days to write all of the tests and that kind of effort can't be justified to the project manager in a work environment. For a personal project why should I spend my precious spare time writing tests when I could be adding more features. A Google search for "hot to start unit testing" returns something like 113,000,000 results. Which tells me that it is quite a popular question but that there are many conflicting opinions about it.
The easiest way that I have found to start unit testing an application is as part of fixing a bug. The method goes something like this.
The key here is that you don't need to have 100% coverage of your code to add and benefit from unit tests. A single test is better than no tests, lots of test cases are better than one. If your code is anything like mine and has lots of bugs when you follow this approach you will very quickly build up a comprehensive set of tests.
You can use the same approach when adding a feature to your application
This is purposely almost exactly the same as the approach for fixing bugs. My final suggestion for introducing testing into your application is to do it when you refactor some code. Take this example I found recently:
class Portfolio(object):
...
def get_portfolio(self, portfolio_code):
...
stmt = """SELECT id FROM portfolios WHERE code=?"""
cursor.execute(stmt, (portfolio_code,))
portfolio_id = cursor.fetchone()[0]
...
def add_value(self, portfolio_code, ...):
...
stmt = """SELECT id FROM portfolios WHERE code=?"""
cursor.execute(stmt, (portfolio_code,))
portfolio_id = cursor.fetchone()[0]
...
I had committed the cardinal sin of repeating some code in a couple of methods. So I split the repeated lines (all 3 of them) out into another method and wrote a couple of tests for that. It had the two benefits I mentioned earlier of making my code easier to understand and making sure each method does one thing and one thing only. Plus I added some much needed unit tests to my application.
This approach to unit testing avoids premature optimisation. If you try and write every possible test, like adding every possible feature to your application you often end up with unnecessary code and unnecessary features.
My advice is to write tests only for the code that you are currently changing. It is not a good idea to think of every possible scenario to test as this often leads to 'analysis paralysis'. You are too worried about thinking of the permutations and combinations to be tested to actually write useful tests. You will also not waste your time writing tests for parts of your application that are rarely executed.
There are some other points to note when you start writing automated unit tests for your code. Unit tested code will look different. There will be more functions and methods and they will each be smaller. Unit testing helps you focus on each method doing one thing well.
When using py.test where do you put your test code? The only hard and fast rule is to keep it separate from your application code. You can put application and test code in the same files but this makes changing your code base harder in the future. Good practice is group like tests together (in classes and/or modules) and clearly identify them. Py.test helps here because it encourages you to start file, class and method names with test. To complete the set I usually put my test code in a directory parallel to my application code called tests.
The other problem people imagine with unit testing is that when you change your application you need to now change two sets of code (application and tests). Whilst this is true fragile tests are a fact of life. When the application functionality changes your tests need to change too. The best approach is the one I described above. If you have tests in an area of your application that you want to modify change the tests first. Then update your code until all of the tests pass again.
Finally, I was listening to issue 182 of the Pod Delusion podcast and they had a piece on the loss of the submarine USS Thresher. It contains a great quote that I'd like to leave you with - "Don't bother conducting tests unless you are willing to listen to the results".
The good news is that there is a lot of it. The bad news is that there is a lot of it to get through ;-)
Just to prove that it is possible to add unit tests to a substantial application read how the LibreOffice project did it.
Here are some introductions to unit testing (mainly using unittest)
Testing tools and libraries