Started in 2008, lldpd is an implementation of IEEE 802.1AB-2005 (aka LLDP) written in C. While it contains some unit tests, like many other network-related software at the time, the coverage of those is pretty poor: they are hard to write because the code is written in an imperative style and tighly coupled with the system. It would require extensive mocking1. While a rewrite (complete or iterative) would help to make the code more test-friendly, it would be quite an effort and it will likely introduce operational bugs along the way.
To get better test coverage, the major features of lldpd are now verified through integration tests. Those tests leverage Linux network namespaces to setup a lightweight and isolated environment for each test. They run through pytest, a powerful testing tool.
pytest in a nutshell§
pytest is a Python testing tool whose primary use is to write tests for Python applications but is versatile enough for other creative usages. It is bundled with three killer features:
- you can directly use the
- you can inject fixtures in any test function, and
- you can parametrize tests.
With unittest, the unit testing framework included with Python, and many similar frameworks, unit tests have to be encapsulated into a class and use the provided assertion methods. For example:
class testArithmetics(unittest.TestCase): def test_addition(self): self.assertEqual(1 + 3, 4)
The equivalent with pytest is simpler and more readable:
def test_addition(): assert 1 + 3 == 4
pytest will analyze the AST and display useful error messages in case of failure. For further information, see Benjamin Peterson’s article.
A fixture is the set of actions performed in order to prepare the system to run some tests. With classic frameworks, you can only define one fixture for a set of tests:
class testInVM(unittest.TestCase): def setUp(self): self.vm = VM('Test-VM') self.vm.start() self.ssh = SSHClient() self.ssh.connect(self.vm.public_ip) def tearDown(self): self.ssh.close() self.vm.destroy() def test_hello(self): stdin, stdout, stderr = self.ssh.exec_command("echo hello") stdin.close() self.assertEqual(stderr.read(), b"") self.assertEqual(stdout.read(), b"hello\n")
In the example above, we want to test various commands on a remote
VM. The fixture launches a new VM and configure an SSH
connection. However, if the SSH connection cannot be established, the
fixture will fail and the
tearDown() method won’t be invoked. The VM
will be left running.
Instead, with pytest, we could do this:
@pytest.yield_fixture def vm(): r = VM('Test-VM') r.start() yield r r.destroy() @pytest.yield_fixture def ssh(vm): ssh = SSHClient() ssh.connect(vm.public_ip) yield ssh ssh.close() def test_hello(ssh): stdin, stdout, stderr = ssh.exec_command("echo hello") stdin.close() stderr.read() == b"" stdout.read() == b"hello\n"
The first fixture will provide a freshly booted VM. The second one will setup an SSH connection to the VM provided as an argument. Fixtures are used through dependency injection: just give their names in the signature of the test functions and fixtures that need them. Each fixture only handle the lifetime of one entity. Whatever a dependent test function or fixture succeeds or fails, the VM will always be finally destroyed.
If you want to run the same test several times with a varying parameter, you can dynamically create test functions or use one test function with a loop. With pytest, you can parametrize test functions and fixtures:
@pytest.mark.parametrize("n1, n2, expected", [ (1, 3, 4), (8, 20, 28), (-4, 0, -4)]) def test_addition(n1, n2, expected): assert n1 + n2 == expected
The general plan for to test a feature in lldpd is the following:
- Setup two namespaces.
- Create a virtual link between them.
- Spawn a
lldpdprocess in each namespace.
- Test the feature in one namespace.
- Check with
lldpcliwe get the expected result in the other.
Here is a typical test using the most interesting features of pytest:
@pytest.mark.skipif('LLDP-MED' not in pytest.config.lldpd.features, reason="LLDP-MED not supported") @pytest.mark.parametrize("clas