Improve docs related with testing changes

Co-authored-by: Laura Barcziová <[email protected]>
Co-authored-by: Matej Focko <[email protected]>

Makefile

@@ -54,10 +54,11 @@ move-stable:
 	[[ -d move_stable_repositories ]] || scripts/move_stable.py init
 	scripts/move_stable.py move-all
 
-# to be run inside VM where the oc cluster && tmt is running!
-# cd containers; make tmt-vagrant-tests instead for testing in the vagrant machine.
+# To be run inside VM where the oc cluster is running!
+# `cd /vagrant; SHARED_DIR=/vagrant make test-deploy` for using it inside the vagrant VM.
+# `SHARED_DIR=/home/tmt/deployment make test-deploy` for using it inside the tmt VM.
 # SHARED_DIR could be /vagrant or /home/tmt/deployment, it depends on the VM where tmt is being run
-# look inside deployment.fmf to find out the value of SHARED_DIR
+# look inside deployment.fmf to find out the value of SHARED_DIR set through tmt
 test-deploy:
 	DEPLOYMENT=dev $(AP) playbooks/generate-local-secrets.yml
 	DEPLOYMENT=dev $(AP) -e '{"user": $(USER), "src_dir": $(SHARED_DIR)}' playbooks/test_deploy_setup.yml

docs/deployment/testing-changes.md

@@ -71,35 +71,57 @@ For more information please refer to [official docs](https://ansible.softwarefac
 
 ### Test Deployment locally with OpenShift Local
 
-For using OpenShift Local you need a _pull secret_, download it here: https://console.redhat.com/openshift/create/local. Save it in a file called `secrets\openshift-local-pull-secret.yml` following this format:
+For using OpenShift Local you need a _pull secret_, download it here: https://console.redhat.com/openshift/create/local. Save it in a file called `secrets/openshift-local-pull-secret.yml` following this format:
 
 ```
 ---
 pull_secret: <<< DOWNLOADED PULL SECRET CONTENT >>>
 ```
 
-Populate the `secrets` dir with all the other secrets.
-You _should use_ your own secrets but if you have access to `stg` secrets
-you can also do:
+[Populate the `secrets` dir (`secrets/{SERVICE}/dev/`) with the other secrets.](secrets#running-a-servicebot-locally)
+
+You can choose if to use a Virtual Machine created by Vagrant or one created by tmt.
+Call a test multiple times and debug it is simpler in a Vagrant VM.
+
+#### Using Vagrant
+
+Create and start the OpenShift Local cluster in a Vagrant VM with (it takes as long as an hour in my X1 ThinkPad):
 
 ```
-DEPLOYMENT=stg make download-secrets
+cd containers; make oc-cluster-create
 ```
 
-Now you can create and start the OpenShift Local cluster (it take as long as an hour) in a Vagrant Virtual Machine with:
+Once OC is up and running you can test the `packit-service` deployment with the command:
 
 ```
-make oc-cluster-create
+cd containers; make tmt-vagrant-test
 ```
 
-And once it is up and running you can test the `packit-service` deployment with the command:
+This command will connect tmt to the Vagrant virtual machine and run the deploy test there (`make test-deploy`).
+You can run the test as many times as you want as long as the virtual machine is up and running and the `crc cluster` is started (`make oc-cluster-up` after every `make oc-cluster-down`).
+You can skip the `tmt` environment and run the test directly inside the VM using `cd containers; make oc-cluster-ssh` and `cd /vagrant && make test-deploy`.
+
+You can destroy the `libvirt` machine with `cd containers; make oc-cluster-destroy` and re-create it again with `cd containers; make oc-cluster-create`.
+
+#### Using tmt
+
+You can test the packit-service deployment using a tmt created local VM with the command:
 
 ```
-make tmt-tests
+make tmt-local-test
 ```
 
-This command will sshed the virtual machine and run the tests there (`make test-deploy`),
-you can run the tests as many time you want as long as the virtual machine is up and running and the `crc cluster` is started (`make oc-cluster-up` after every `make oc-cluster-down`).
-You can skip the `tmt` environment and run the test directly inside the VM using `make oc-cluster-ssh` and `cd /vagrant && make test-deploy`.
+It is quite hard to change a test inside a tmt created VM and debug it.
+But, in case you need it this is a list of commands that can be handy:
 
-You can destroy the `libvirt` machine with `make oc-cluster-destroy` and re-create it again with `make oc-cluster-create`.
+```
+tmt run --id deployment --until execute
+tmt run --id deployment prepare --force
+tmt run --id deployment login --step prepare:start
+tmt run --id deployment execute --force
+tmt run --id deployment login --step execute:start
+tmt run --id deployment finish
+tmt clean runs
+tmt clean guests
+virsh list --all
+```