Skip to content

Commit

Permalink
test: introduce an integration test suite (#459)
Browse files Browse the repository at this point in the history
* ci: build on alt-archs as well

* test: introduce an integration test suite

Since we already use Packit, let's utilize its second part as well and
add a TMT [0] based integration test suite.

This PR add a second job to the existing Packit configuration, which
then sends the just built RPMs to Testing Farm [1] that executes all
selected tests (which currently means all discovered tests).

To demonstrate the functionality a bit, this PR also adds a simple test
case for #439.

[0] https://tmt.readthedocs.io/en/stable/overview.html
[1] https://docs.testing-farm.io/Testing%20Farm/0.1/index.html
  • Loading branch information
mrc0mmand authored Jun 10, 2024
1 parent d89c360 commit e89a5db
Show file tree
Hide file tree
Showing 8 changed files with 136 additions and 1 deletion.
13 changes: 12 additions & 1 deletion .packit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,15 @@ jobs:
- job: copr_build
trigger: pull_request
targets:
- fedora-all
- fedora-all-aarch64
- fedora-all-ppc64le
- fedora-all-s390x
- fedora-all-x86_64

- job: tests
trigger: pull_request
fmf_path: test/integration/
tmt_plan: upstream-ci
targets:
- fedora-all-aarch64
- fedora-all-x86_64
1 change: 1 addition & 0 deletions test/integration/.fmf/version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1
54 changes: 54 additions & 0 deletions test/integration/README
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# polkit integration test suite

polkit's integration test suite uses TMT (Test Management Tool [0]) to organize and run tests. Since TMT
offers a _lot_ of features, this document pinpoints the most _interesting_ ones to get stuff up and running
quickly.

## How to contribute

Creating a new test case is pretty simple:

```
$ cd test/integration
$ tmt test create --template=shell test/name
Test directory '/home/.../polkit/test/integration/test/name' created.
Test metadata '/home/.../polkit/test/integration/test/name/main.fmf' created.
Test script '/home/.../polkit/test/integration/test/name/test.sh' created.
```

The newly created `test.sh` will be the actual test case, and `main.fmf` contains the test metadata, including
test summary & description, test dependencies, runtime, and so on. See [1] for more details.

After tweaking the test metadata it's usually a good idea to run `tmt lint test/name` to make sure that the
configuration is still valid:

```
$ tmt lint test/name
/test/name
pass C000 fmf node passes schema validation
pass C001 summary key is set and is reasonably long
...
```

To check if the test itself works as expected you can use `tmt run`:

```
$ cd test/name
$ tmt run -vvv --all provision --how local tests --name .
...
total: 1 test passed
```

The `tmt run` command is _very_ customizable (as is the rest of `tmt`). In this particular example we tell it
to run all steps (`--all`) and override the `provision` and `tests` steps to run just one particular test on
the local machine. As in previous cases, check the `tmt` documentation [0] and examples [2] for more details.

## Links

[0] https://tmt.readthedocs.io/en/stable/overview.html

[1] https://tmt.readthedocs.io/en/stable/spec/tests.html

[2] https://tmt.readthedocs.io/en/stable/examples.html#run

<!-- vim: set syntax=markdown tw=110 : -->
2 changes: 2 additions & 0 deletions test/integration/main.fmf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
require:
- polkit
11 changes: 11 additions & 0 deletions test/integration/plans/upstream-ci.fmf
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# vi: set sw=2 ts=2 et ft=yaml tw=80:

# This plan discovers and executes all (enabled) integration tests

summary: Upstream integration test suite
/:
inherit: false
discover:
how: fmf
execute:
how: tmt
2 changes: 2 additions & 0 deletions test/integration/systemd/main.fmf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
summary: Check integration with systemd
test: ./test.sh
8 changes: 8 additions & 0 deletions test/integration/systemd/rules/start-restart-stop-unit.rules
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
polkit.addRule(function(action, subject) {
if (action.id == "org.freedesktop.systemd1.manage-units") {
if (subject.user == "polkit-testuser" && action.lookup("unit") == "start-restart-stop.service" &&
(action.lookup("verb") == "restart" || action.lookup("verb") == "stop" || action.lookup("verb") == "start")) {
return polkit.Result.YES;
}
}
});
46 changes: 46 additions & 0 deletions test/integration/systemd/test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
#!/bin/bash

set -eux
set -o pipefail

TEST_RULES="$PWD/rules"
TEST_USER="polkit-testuser"

at_exit() {
set +e

: "Cleanup"
userdel -rf "$TEST_USER"
rm -f /etc/polkit-1/rules.d/99-test.rules
systemctl restart polkit
}

trap at_exit EXIT

: "Setup"
mkdir -p /run/systemd/system/
useradd "$TEST_USER"
# Close stdin, so we get an instant error (Interactive authentication required) instead of having to deal
# with an interactive authentication prompt
exec 0<&-

: "Allow $TEST_USER to start/restart/stop a simple systemd unit"
# Use `systemctl edit --full ...` in the future
cat >/run/systemd/system/start-restart-stop.service <<EOF
[Service]
Type=oneshot
ExecStart=true
EOF
systemctl daemon-reload
# Copy the test polkit rule in place
cp -fv "$TEST_RULES/start-restart-stop-unit.rules" /etc/polkit-1/rules.d/99-test.rules
systemctl restart polkit
# Following systemctl invocations should not trigger polkit's authentication prompt
sudo -u "$TEST_USER" systemctl start start-restart-stop.service
sudo -u "$TEST_USER" systemctl restart start-restart-stop.service
sudo -u "$TEST_USER" systemctl stop start-restart-stop.service
# But these ones should
(! sudo -u "$TEST_USER" systemctl mask start-restart-stop.service)
(! sudo -u "$TEST_USER" systemctl restart systemd-journald.service)
# Cleanup
rm -f /etc/polkit-1/rules.d/99-test.rules

0 comments on commit e89a5db

Please sign in to comment.