From Univention Wiki
ucs-test is used to check UCS installations for correct working and configuration. This includes tests like the following:
- files have the right access permissions
- required packages are installed
- services are running
- updates are working
Until UCS-2.4 ucs-test was mainly used internally, but since UCS-3.0 the framework can be used and extended by external developers and administrators as well.
- 1 Structure
- 2 Packages
- 3 Usage
- 4 Creating new tests
- 5 Quality
The tests are categorized in sections, which are represented as sub-directories like /usr/share/ucss-test/00_section/. Every directory must begin with a two digit number followed by a name, separated by an underscore character. Every section should be packaged in a separate (binary) Debian package to allow picking only relevant tests.
For every test a single executable file exists, which implements the test. The name of the file must begin with a two digit number, followed by a short description. No file extension (.py, .sh, ...) should be used. In the hash-bang-line #!/usr/share/ucs-test/runner should be used instead of using /bin/bash or /usr/bin/python directly. Is interpreter asserts several things:
- The current working directory is changed to the directory containing the script.
- Meta data information from the test case file are used to check the required system role, check for required packages and versions. The test is only run if those pre-conditions are met.
- The (integer) return value of the test script is converted to a human readable message.
Next to the test files some shell and python libraries are provided below /usr/share/ucs-test/lib/, which can be re-used by several tests.
/usr/sbin/ucs-test is a program to run several tests at once. It collects their output and creates some summary information.
- This package contains the minimum required files like runner and ucs-test.
- This package contains the optional shared libraries used by several other packages.
- Each package contains the tests of the different sections.
The ucs-test-libs package provides several libraries for common tasks, such as creating and modifying UDM objects, handling UCR variables, or common functions to simplify writing robust tests. Shell libraries should be included as
. "$TESTLIBPATH/name.sh" || exit 137
while Python libraries should be imported as regular modules like
Currently the following libraries are shipped:
- Output helpers, software checks, version checks, test helpers
- Simplify handling UDM computers
- Simplify handling UDM container
- Simplify handling UDM groups
- Directly interact with LDAP
- Execute commands remotely on domaincntroller_master
- Provide function to generate random names for different requirements
- Simplify handling UDM shares
- Wrapper around UCR to restore values on exit
- Simplify handling UDM users
Tests can be run in two ways. Either directly using their fully qualified name (/usr/share/ucs-test/00_section/00_test), or using ucs-test to run several tests together. ucs-test supports several options to select a subset of all installed tests, which are described in its manual page (man ucs-test).
Creating new tests
In its simplest form a test contains only some commands, which return 0 on success. Any other return value is interpreted as a failure.
#!/usr/share/ucs-test/runner bash ## desc: Test if /etc/shadow exists ## exposure: safe [ -f '/etc/shadow' ] # vim: set filetype=sh tabstop=4 :
- In the 1st line /usr/share/ucs-test/runner is used instead of /bin/bash. This wrapper prints some more information in addition to the return value of the tests. The language used for the implementation of the test is passed as the sole parameter to this wrapper. Next to bash, python and python2.6 any other interpreter might be used as well as long as the fully qualified path to the interpreter is given.
- In the 2nd line a description is given using the meta data syntax. This description is included in the output next to the return value. The meta data block must be located at the beginning of the file right after the hash-bang line without any intermediate lines. All lines must pre prefix by ## .
- The 3rd line declares that the test is safe to be run on any system, even systems already in production.
- The 4th line implements the tests.
The meta data information consists of consecutively lines, which directly follow the hash-bang line. The block ends at the first line not prefixed by ## . Without that prefix the format follows YAML. This allows other programs to parse that information as well and also allows custom data (severity, author, ...) to be added. ucs-test uses the following entries:
A text used as a description for this test, which is displayed together with the test result when the test is run. The text can span multiple lines, which allows adding detailed information. How much of the description is displayed depends on the chosen output format.
## desc: A short text ## desc: | ## A shot text for displaying. ## A longer text, which describes the test in more detail.
A list if bug numbers from the Univention Bugzilla. Depending on the chosen output format (for example html), reports will include links to the referenced bugs.
## bugs:  ## bugs: ## - 23527
A list of ticket numbers from the Univention OTRS. Depending on the chosen output format (for example html), reports will include links to the referenced tickets.
## otrs: [2012031912061xxx]
A mapping from UCS versions number (major.minor-patchlevel) to the strings found, fixed und skip. This allows to differentiate the return values and to distinguish between tests, which are for known not-yet-fixed bugs, and tests, which should be fixed but fail again.
- the bug is relevant starting from that version.
- the bus was fixed in this version and should not trigger.
- the test should be skipped since that version.
## version: ## 2.0-0: found ## 2.1-0: fixed ## 3.0-0: skip
A list of tags, which can be used to group tests. This can be used to either run only a specific subset of all installed tests or prevent some tagged tests from running. The name for tags can be chosen freely, but the following words have special meaning:
- SKIP, WIP
- The test will be skipped by default, because — for example — the implementation is still work in progress.
- Marks the tests as a basic test, which should be run on all systems to assert a sound installation.
## tags: [WIP] ## tags: ## - my-tag
A list of UCS role names. The test is only run on systems, which match the role name explicitly listed here. The default is to allow the test to run on all roles.
## roles: ## - domaincontroller_master ## - domaincontroller_backup ## - domaincontroller_slave ## - memberserver ## - basesystem ## - managedclient ## - mobileclient ## - fatclient
Inverse list for UCS role names. The test is skipped on systems, which match the role name listed here. Overlaps with roles will abort the test.
## roles: [basesystem]
Requires or prohibits the system to be joined. Otherwise the test is not run. By default the join status is irrelevant and the test is run on all systems.
## join: true
A mapping of component names to a boolean value, which requires the named component to be either installed or to be deactivated. By default no components are required and the test is run on all systems.
## components: ## tcs: true ## dvs: false
A list of Debian package dependencies, which must be fulfilled. Otherwise the test is not run on such a system. Each entry follows the syntax used by Debian dependencies.
## packages: ## - dpkg-dev (>= 1.15) ## - apache2 | apache2-mpm-prefork
Normally dependencies should be specified using the Debian debian/control file. But in some cases tests need special software, which is not wanted on normal systems. This mechanism allows the user to still install the full ucs-test-* section package without forcing him to install several other packages only relevant for corner case tests. By using this mechanism of ucs-test, tests still missing some dependent packages are skipped.
A string consisting of one of the words safe, careful or dangerous, optionally followed by the md5sum over the file without the meta data block separated by a blank. This is used to classify tests in different categories:
- tests of read data and never modify anything on the system. These tests can be run on production systems without danger of modifying that system or losing services.
- test do modify the system (create files, restart services, change configurations), but revert all changes back to the original state at the end of the test, regardless of if the test succeeds or fails.
- the test has side effects, which might change this and other systems in a way unfit for production systems. This includes — for example — creating users and groups in LDAP, re-configuring essential services like LDAP.
The default is dangerous.
As an extra precaution against unwanted modification an optional MD5 checksum can be supplied. The test is skipped in case the specified checksum does not match the actual checksum.
## exposure: careful ## exposure: safe 25f9fc4bf6e01167a0cc6c84309572b7
The MD5 checksum can be computed for example by running
sed -e '/^## /d' "/usr/share/ucs-test/tests/00_section/00_test" | md5sum
In its simplest case a test should return 0 for success and 1 on failure. For backward compatibility to previous versions of ucs-test several other special values can be returned as well to provide more detailed information. More information is available in the source code or internally from UCS-Test until UCS-2.4.
ucs-test should be runnable on all systems, ranging from development machines to systems in production. Because of this requirement test developers should be extremely careful before modifying system configuration or data. To protect production systems from accidental destruction by running tests, ucs-test requires tests to follow the following criteria:
- a test should just test one thing, so a failed tests has a clear conclusion.
- a test should return stable results, so running the same test multiple times returns the same result each time.
- a test should not modify the system. If anything is changes, it must be changed back regardless if the test succeeds or fails.
To enforce this ucs-test by default refrains from running tests with an #exposure level other then safe.