Tests¶
Running tests¶
Using tox¶
The preferred way to run tests is to use tox. It will take care of everything and run the tests on all supported Python versions (each in its own virtualenv) and all target operating systems :
$ pip install -r dev-requirements.txt
$ tox
Tox will also build the Sphinx documentation, so it will tell you about any reStructuredText syntax errors.
Extra options after a --
on the command line will be passed to the
py.test test runner. For example, to stop immediately
after the first failure:
$ tox -- -x
Or to only run tests whose name matches apache
:
$ tox -- -k apache
Note
If tox ever gives you trouble, you can ask it to recreate its virtualenvs
by using the -r
(or --recreate
) option. Alternatively, you can start
over completely by removing the .tox
directory.
Using py.test¶
If you want to use py.test
directly, you will first need to install the test
dependencies. You will also need to install fabtools itself in development
mode (also called editable mode):
$ pip install pytest mock
$ pip install -e .
$ py.test
Unit tests¶
The goal of the unit tests is to test the internal logic of fabtools functions, without actually running shell commands on a target system.
Most unit tests make use of the mock library.
Functional tests¶
The goal of the functional tests is to test that fabtools functions have the expected effect when run against a real target system.
Functional tests are contained in the fabtools/tests/functional_tests/
folder.
Requirements¶
Running functional tests requires Vagrant and VirtualBox to launch the virtual machines against which the tests will be run.
If Vagrant is not installed, the functional tests will be skipped automatically and pytest will show a warning message.
Target boxes¶
The default tox configuration will run the functional tests using both Python 2.6 and 2.7, against a specific list of vagrant boxes. These boxes will be downloaded from Atlas (formerly Vagrant Cloud) when needed if they’re not already installed on your computer.
Target OS | Vagrant Box Name |
---|---|
centos_6_5 |
chef/centos-6.5 |
debian_6 |
chef/debian-6.0.10 |
debian_7 |
chef/debian-7.8 |
debian_8 |
debian/jessie64 |
ubuntu_12_04 |
hashicorp/precise64 |
ubuntu_14_04 |
ubuntu/trusty64 |
A tox environment name is the combination of the Python version
(either py26
or py27
) and a target operating system.
You can use tox -l
to get the list of all test environments.
You can use the -e
option to run tests in one or more specific
environments. For example, you could run the tests using Python 2.7
only, against both Ubuntu 12.04 and 14.04 boxes
$ tox -e py27-ubuntu_12_04,py27-ubuntu_14_04
Skipping the functional tests¶
To run the unit tests only, you can use the none
target:
$ tox -e py26-none,py27-none
Using a specific Vagrant box¶
If you want to run the tests with a specific Vagrant box, you can use
the FABTOOLS_TEST_BOX
environment variable and the none
target:
$ export FABTOOLS_TEST_BOX='mybox'
$ tox -e py27-none
Using a specific Vagrant provider¶
If you want to run the tests with a specific Vagrant provider, you can use
the FABTOOLS_TEST_PROVIDER
environment variable:
$ export FABTOOLS_TEST_BOX='vmware_box'
$ export FABTOOLS_TEST_PROVIDER='vmware_fusion'
$ tox -e py27-none
Debugging functional tests¶
When you’re working on a functional test, sometimes you’ll want to manually inspect
the state of the Vagrant VM. To do that, you can prevent it from being destroyed
at the end of the test run by using the FABTOOLS_TEST_REUSE_VM
environment
variable:
$ export FABTOOLS_TEST_REUSE_VM=1
$ tox -e py27-ubuntu_14_04 -- -x -k apache
$ cd fabtools/tests/functional_tests
$ vagrant ssh