Unit-testing non-deterministic functionality in Python

Written by Johannes on August 28, 2013 Categories: Python, Test Driven Development Tags: , ,

Your ads will be inserted here by

Easy AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

In a certain part of our REST-API we expose object ids via randomized unique id (UUID4). While this is convenient in helping to prevent race conditions that might have occured with incrementing integers in the relevant part of the code, randomness always is somewhat of a pain for unit testing. After all, a key necessity to unit testing is being able to predict the results of function calls.

I solved the problem by using a thread local, which I called test_registry. I am now able to tack any data onto this thread local from my tests, which in particular gives me the ability to provide mock random data. In the case of uuid creation it looked something like this.

# core/api.py ## core module code
from threading import local
test_registry = local()
test_registry.test = False
 
# model.py ## model logic code importing uuid.uuid4
from core.app import test_registry
 
def uuid4():
    from uuid import uuid4 as uuid4_orig
    if test_registry.test:
        return test_registry.mock_guids.pop()
    return uuid4_orig()
 
 
# test.py
import unittest
import uuid
from core.app import test_registry
 
class MyTestCase(unittest.TestCase):
    def setUp(self):
        test_registry.test = True
        test_registry.mock_guids = [uuid.uuid4() for i in xrange(4)]
        self.mock_guids = list(mock_guids)
 
    def test_mock_random(self):
        from model import uuid4
        for i in xrange(4):
             self.assertEqual(uuid4(), self.mock_guids.pop())

A few additional thoughts on this: for the sake of your tests: make sure you fail the tests when you run out of mock randomness. In the example above test_registry.mock_guids.pop() will raise an IndexError once the list of uuids has been depleted. If you don’t deliberately fail your tests when running out of mock randomness, you might accidentally write non-deterministic tests, that will randomly fail or, even worse: randomly succeed. Testing is about predictability so keep that in mind.

A second thought: use this with care! 1.) I’m using thread locals here. IMHO this is ok, because the purpose is very clearly delimited and won’t leak into business logic relying on them. 2.) Security: take extra cautions when providing overrides for security-related randomness. Having predictably encryption keys is something you don’t want to have in your software.

No Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre user="" computer="" escaped="">