openQA is an automated test tool that makes it possible to test the whole installation process of an operating system. It’s free software released under the GPLv2 license. The source code and documentation are hosted in the os-autoinst organization on GitHub.
This document provides the information needed to start contributing to the openQA development improving the tool, fixing bugs and implementing new features. For information about writing or improving openQA tests, refer to the Tests Developer Guide. In both documents it’s assumed that the reader is already familiar with openQA and has already read the Starter Guide. All those documents are available at the official repository.
As mentioned, the central point of development is the os-autoinst organization on GitHub where several repositories can be found:
-
openQA containing documentation, server, worker and other support scripts.
-
os-autoinst with the standalone test tool.
-
os-autoinst-distri-opensuse containing the tests used in http://openqa.opensuse.org
-
os-autoinst-needles-opensuse with the needles associated to the tests in the former repository.
-
os-autoinst-distri-example with an almost empty set of tests meant to be used to start writing tests (and creating the corresponding needles) from scratch for a new operating system.
As in most projects hosted on GitHub, pull request are always welcome and are the right way to contribute improvements and fixes.
-
Every commit is checked by Travis CI as soon as you create a pull request but you should run the openQA tests locally, i.e. before every commit call:
./script/tidyto ensure your perl code changes are consistent with the style rules
-
You may also run local tests on your machine or in your own development environment to verify everything works as expected. Call:
make testfor unit and integration tests.
-
For git commit messages use the rules stated on How to Write a Git Commit Message as a reference
-
Every pull request is reviewed in a peer review to give feedback on possible implications and how we can help each other to improve
If this is too much hassle for you feel free to provide incomplete pull requests for consideration or create an issue with a code change proposal.
But developers willing to get really involved into the development of openQA or people interested in following the always-changing roadmap should take a look at the openQAv3 project in openSUSE’s project management tool. This Redmine instance is used to coordinate the main development effort organizing the existing issues (bugs and desired features) into 'target versions'.
Currently developers meet in IRC channel #opensuse-factory and in a daily jangouts call of the core developer team.
In addition to the ones representing development sprints, two other versions are always open. Easy hacks lists issues that are not specially urgent and that are considered to be easy to implement by newcomers. Developers looking for a place to start contributing are encouraged to simply go to that list and assign any open issue to themselves. Future improvements groups features that are in the developers' and users' wish list but that have little chances to be addressed in the short term, either because the return of investment is not worth it or because they are out of the current scope of the development.
openQA and os-autoinst repositories also include test suites aimed at preventing bugs and regressions in the software. codecov is configured in the repositories to encourage contributors to raise the tests coverage with every commit and pull request. New features and bug fixes are expected to be backed with the corresponding tests.
Everything in openQA, from os-autoinst to the web frontend and from the tests to the support scripts is written in Perl. So having some basic knowledge about that language is really desirable in order to understand and develop openQA. Of course, in addition to bare Perl, several libraries and additional tools are required. The easiest way to install all needed dependencies is using the available os-autoinst and openQA packages, as described in the Installation Guide.
In the case of os-autoinst, only a few CPAN modules are required. Basically Carp::Always, Data::Dump. JSON and YAML. On the other hand, several external tools are needed including QEMU, Tesseract and OptiPNG. Last but not least, the OpenCV library is the core of the openQA image matching mechanism, so it must be available on the system.
The openQA package is built on top of Mojolicious, an excellent Perl framework for web development that will be extremely familiar to developers coming from other modern web frameworks like Sinatra and that have nice and comprehensive documentation available at its home page.
In addition to Mojolicious and its dependencies, several other CPAN modules are required by the openQA package. For a full list of hard dependencies, see the file DEPENDENCIES.txt at the root of the openQA repository. Some additional modules could be required when using a database engine other than the default SQLite.
As already mentioned, openQA relies on a database engine to store the information. PostgreSQL, MySQL and SQLite3 are supported, with the latter being the default option.
Lastly, png2theora (part of the Theora project) is used to create a video from the screenshots generated by os-autoinst.
As stated in the previous section, every feature implemented in both packages should be backed by proper tests. Test::More is used to implement those tests. As usual, tests are located under the /t/ directory. In the openQA package, one of the tests consists of a call to Perltidy to ensure that the contributed code follows the most common Perl style conventions.
During the development process there are cases in which the database schema needs to be changed. After modifying files in lib/OpenQA/Schema/Result there are some steps that have to be followed so that new database instances and upgrades include those changes.
-
First, you need to increase the database version number in the
$VERSIONvariable in the lib/OpenQA/Schema.pm file. Note that it’s recommended to notify the other developers before doing so, to synchronize in case there are more developers wanting to increase the version number at the same time. -
Then you need to generate the deployment files for new installations, this is done by running ./script/initdb --prepare_init.
-
Afterwards you need to generate the deployment files for existing installations, this is done by running ./script/upgradedb --prepare_upgrade. After doing so, the directories dbicdh/$ENGINE/deploy/<new version> and dbicdh/$ENGINE/upgrade/<prev version>-<new version> for SQLite and PosgreSQL should have been created with some SQL files inside containing the statements to initialize the schema and to upgrade from one version to the next in the corresponding database engine.
-
And finally, you need to create the fixtures files. Under dbicdh/_common/deploy, rename the directory of the (previous) latest version to the new version and do the necessary changes (if any). Then, under dbicdh/_common/upgrade create a <prev_version>-<new_version> directory and put some files there with SQL statements that upgrade the fixtures. Usually a diff from the previous version to the new one helps to see what has to be in the upgrade file.
The above steps are executed in the developer’s system. Once openQA is installed in a production server, you should run either ./script/initdb --init_database or ./script/upgradedb --upgrade_database to actually create or upgrade a database.
Fixtures (initial data stored in tables at installation time) are stored in files into the dbicdh/_common/deploy/_any/<version> and dbicdh/_common/upgrade/<prev_version>-<next_version> directories.
You can create as many files as you want in each directory. These files contain SQL statements that will be executed when initializing or upgrading a database. Note that those files (and directories) have to be created manually and they shouldn’t create a transaction, since each file is already automatically executed in its own transaction (so that changes are rolled back if there’s any problem) and sqlite doesn’t support nested transactions.
Executed SQL statements can be traced by setting the DBIC_TRACE environment variable.
export DBIC_TRACE=1It can be necessary during development to change the config files in etc/. For example you have to edit etc/openqa/database.ini to use another database. Or to increase the log level it’s useful to set the loglevel to debug in etc/openqa/openqa.ini.
To avoid these changes getting in your git workflow, copy them to a new directory and set OPENQA_CONFIG in your shell setup files.
cp -ar etc/openqa etc/mine
export OPENQA_CONFIG=$PWD/etc/mineNote that OPENQA_CONFIG points to the directory containing openqa.ini, database.ini, client.conf and workers.ini.
-
Install PosgreSQL - under openSUSE the following package are required: postgresql-server postgresql-init
-
Start the server: systemctl start postgresql
-
The following steps need to be done by the user postgres: su - postgres
-
Create user: createuser your_username where your_username must be the same as the UNIX user you start your local openQA instance with.
-
Create database: createdb -O your_username openqa
-
The next steps must be done by the user you start your local openQA instance with.
-
Import dump: pg_restore -c -d openqa path/to/dump
-
Configure openQA to use PostgreSQL as described in the section Other database engines of the installation guide. User name and password are not required.
OpenQA comes with three authentication modules providing authentication methods: OpenID, iChain and Fake (see User authentication).
All authentication modules reside in lib/OpenQA/Auth directory. During OpenQA start, [auth]/method section of /etc/openqa/openqa.ini is read and according to its value (or default OpenID) OpenQA tries to require OpenQA::WebAPI::Auth::$method. If successful, module for given method is imported or the OpenQA ends with error.
Each authentication module is expected to export auth_login and auth_logout functions. In case of request-response mechanism (as in OpenID), auth_response is imported on demand.
Currently there is no login page because all implemented methods use either 3rd party page or none.
Authentication module is expected to return HASH:
%res = (
# error = 1 signals auth error
error => 0|1
# where to redirect the user
redirect => ''
);Authentication module is expected to create or update user entry in OpenQA database after user validation. See included modules for inspiration.
It is possible to customize the openQA base directory by setting the environment variable OPENQA_BASEDIR. The default value is /var/lib.
To execute the testsuite locally, use make test. It is also possible to run a particular test for example prove t/api/01-workers.t.
To run UI tests the package perl-Selenium-Remote-Driver is required. Note that the version provided by Leap 42.2 is too old. The version from the repository devel-languages-perl can be used instead.
To use ChromeDriver instead of PhantomJS the environment variable SELENIUM_CHROME can be set.