Added acceptance tests container #6
Conversation
|
|
@scollazo this is being designed with CI in mind where human interaction isn't a priority. To my knowledge Xvfb seemed simpler and still had the ability to take screenshots. Wouldn't that be enough? |
|
vnc allows for things like recording videos from the session in an easyer way, and also it would be useful when using this in a standalone way, not hooked to ci/cd |
|
Can there be a common image, and then xvnc or alternative added in a
separate layer? So we end up with 2 images to use, but also a third
without either option included?
…On Oct 23, 2017 1:33 PM, "Santiago Rodríguez" ***@***.***> wrote:
vnc allows for things like recording videos from the session in an easyer
way, and also it would be useful when using this in a standalone way, not
hooked to ci/cd
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AC8O3fQdRJ0pXhTPd8IhAdtiGQDYfM_dks5svPgQgaJpZM4PhIqy>
.
|
|
These are really nice ideas, having more than one option would be great. I wonder though if we should focus first on getting the simplest way possible working (afaik that's Xvfb), have CI running it automatically for us and then look into your ideas as separate issues? e.g. At this point I think that the biggest change that @mamedin's work needs is to make some tweaks in the entry point of the container so we can run the tests exactly when we want to without having to edit the script and rebuild the image. E.g. I suggest to replace $ ansible-playbook ../playbooks/download-sources.yml
$ make create-volumes
$ docker-compose up -d --build
$ make bootstrap
$ make run-acceptance-testsThe problem with the existing entry point is that it doesn't take inputs: #!/bin/sh
rm -rf /tmp/.X99-lock /tmp/.X11-unix/X99 &
Xvfb -ac :99 -screen 0 1920x1080x8 -nolisten tcp &
cd /home/archivematica/test/
./no_test.sh
#./firefox_am17_tests.sh
#./chrome_am17_tests.shWe could replace it with something like the following: #!/usr/bin/env bash
set -ex
if [ $# -eq 0 ]; then
echo "No arguments supplied."
exit 0
fi
firefox=false
chrome=false
for arg in "$@"
do
case $arg in
"firefox" )
firefox=true;;
"chrome" )
chrome=true;;
"all" )
firefox=true; chrome=true;;
esac
done
if [ "$firefox" = true ]; do
./firefox_am17_tests.sh
done
if [ "$chrome" = true ]; do
./chrome_am17_tests.sh
doneThe implementation of That |
sevein
force-pushed
the
dev/acceptance-tests
branch
5 times, most recently
from
c17b73e
to
029f3cb
Compare
|
I've rebased this, now it's using git submodules. I've also made some changes on top of @mamedin's work. Instead of building our own images which proved to be really hard, I've added a new These containers seem to be pretty flexible and they can be complemented with other alternatives that they provide with VNC and virtual desktop environments. Also they use recent versions of the browsers and web drivers (geckodriver and chromedriver). More details here: https://github.com/SeleniumHQ/docker-selenium. Please revisit the diff for more details. I've added some docs. They don't explain how to use VNC |
compose/README.md
Outdated
|
|
||
| There is an extra Compose file called `docker-compose.acceptance-tests.yml` that defines some extra services to run our acceptance tests, e.g. Selenium Standalone Server, Selenium Chrome Node or Selenium Firefox Node. In order to run the tests use the following command: | ||
|
|
||
| $ make at-run |
There was a problem hiding this comment.
I need to support passing arguments to behave, e.g.:
$ make ARGS="--tags=@wip" at-run
| CHROME_VERSION: "google-chrome-stable" | ||
| CHROME_DRIVER_VERSION: "latest" | ||
| HUB_PORT_4444_TCP_ADDR: "selenium-hub" | ||
| HUB_PORT_4444_TCP_PORT: "4444" |
There was a problem hiding this comment.
I think it defaults to hub (instead of selenium-hub), but I can rename the link below.
There was a problem hiding this comment.
Like I do in the archivematica-acceptance-tests service.
sevein
force-pushed
the
dev/acceptance-tests
branch
4 times, most recently
from
d85589d
to
577f0a2
Compare
|
Rebased. |
sevein
force-pushed
the
dev/acceptance-tests
branch
3 times, most recently
from
ac2a428
to
aa435d7
Compare
|
Rebased and squashed. |
|
At 70b45f3 I was able to run |
| environment: | ||
| FIREFOX_VERSION: "47.0.1" |
There was a problem hiding this comment.
Is this here for a reason? I noticed that https://hub.docker.com/r/selenium/node-firefox/~/dockerfile/ takes a FIREFOX_VERSION but it's a build argument not an environment variable. Is this being used somewhere else?
| - "nginx" | ||
| shm_size: 2g # See https://goo.gl/dkZzHh | ||
| ports: | ||
| - "62102:5900" |
There was a problem hiding this comment.
Could you add the new ports to the README?
| ports: | ||
| - "62102:5900" | ||
| environment: | ||
| FIREFOX_VERSION: "47.0.1" |
There was a problem hiding this comment.
Is this here for a reason? I noticed that https://hub.docker.com/r/selenium/node-firefox/~/dockerfile/ takes a FIREFOX_VERSION but it's a build argument not an environment variable. Is this being used somewhere else?
There was a problem hiding this comment.
Ha. I don't even know if this does anything. All I know is that I got the test to pass when this was set. I'll rerun them without this and see if they pass.
|
TODOs:
|
mamedin
force-pushed
the
dev/acceptance-tests
branch
from
152a98f
to
8732bd3
Compare
|
About the TODO checklist, I think that we are done. Please, review the latest 4 commits. The entrypoint commit is related to artefactual-labs/archivematica-acceptance-tests#27 |
sevein
force-pushed
the
dev/acceptance-tests
branch
from
8732bd3
to
9949aaa
Compare
This pull request tries to fix #2. It was originally submitted by @mamedin but it has received contributions from other developers.
The goal is to make possible to run the acceptance tests in our Docker Compose environment. We've added a new
docker-compose.acceptance-tests.ymlconfiguration file that adds four new services:selenium-hub(Java daemon)selenium-firefox(Firefox node)selenium-chrome(Chrome node)archivematica-acceptance-tests(behave + tests)behaveuses itswebdriverto communicate withselenium-hub. The hub receives the tests to be executed along with information on which browser and platform where the test should be run. The hub will use this information and delegate to a node that can service those needs.It's easier to understand it just by looking at this simple behave test that we've added to verify that the selenium infra is working: https://github.com/artefactual-labs/archivematica-acceptance-tests/blob/acceptance-tests-docker/simplehubtest.py. This test can be executed with
make test-at-check.Other
test-at-*targets have been added to ourMakefile. Runmake helpto obtain the full list.