Tempest: An OpenStack Integration test suite

Integration Testing:

• Integration Testing is the phase in software testing in which individual software modules are combined and tested as a group.
• It occurs after unit testing and before system testing.
• Tests the actual functionality of the application/software you have written.

Why wasting time in integration tests when I have an option to check the functionality manually?

• Because every time you add a new feature you will have to check and re-check the functionalities manually which is tedious, time-consuming and error-prone.
• What if you have thousands of features, certainly you can’t afford to check if them one by one.
• If you claim that your software is platform-independent. How will you make sure that all the functionalities of your software are intact on multiple platforms? That’s where a ‘test-suite’ comes into picture. You can create a dedicated test environment to run your test-suite across different platforms adhering to your scale and stress needs.

I have written unit tests properly. Why should I write integration tests?

• Because unit-tests test the source code you have written, the semantic and the syntax, not the actual functionality of your software.
• Integration tests make sure that your source code, when integrated with all your components, serves the real purpose.
• With addition of a new feature you can just re-trigger the suite and hence make sure that your new feature doesn’t break any existing functionality.

What is tempest?

• One of Tempest’s prime function is to ensure that your OpenStack cloud works with the OpenStack API as documented. The current largest portion of Tempest code is devoted to test cases that do exactly this.
• It’s also important to test not only the expected positive path on APIs, but also to provide them with invalid data to ensure they fail in expected and documented ways. Over the course of the OpenStack project Tempest has discovered many fundamental bugs by doing just this.
• In order for some APIs to return meaningful results, there must be enough data in the system. This means these tests might start by spinning up a server, image, etc., then operating on it.
• Currently tempest has support to test both APIs and CLIs.

How to run the tempest tests?

• There are multiple ways to run the tests. You have a choice to run the entire test suite, all the tests in a directory, a test module or a single tests.
• The documents describes how to run the tempest tests using nosetests.
• Install the basic dependencies:
  • apt-get install pip (if you have Ubuntu)
  • pip install nosetests
  • pip install testresources
  • git clone http://github.com/openstack/tempest
    
• Go to the directory tempest and execute the tests with the following command;
  • nosetests -sv tempest.tests.identity.admin.test_services.py [ This would run all the tests inside the module test_services.py]
  • nosetests -sv tempest.tests.identity.admin.test_services:ServicesTestJSON.test_create_get_delete_service [This runs a specific tests inside the class ServiceTestJSON of the module test_services.py]

Basic Architecture

• Tempest tests can be executed from either outside the openstack setup or from the openstack controller itself.
• The API tests are based on rest. (http://ilearnstack.com/2013/04/25/understanding-rest-implementation/)
  • The tempest has a rest client implemented in the suite which takes care of the REST operations.
  • All the user inputs which are performing the rest operationon nova apis are read from tempest.conf file.
  • The suite first make a rest call to the keystone api to get the auth token and tenant details for authorization and authentication.
  • Using the auth token obtained from the response, the tests performs the required operations.

More to follow…

• Debugging using eclipse..
• Extending tempest framework for your project..