-
Notifications
You must be signed in to change notification settings - Fork 17
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add wraith for visual regression testing
Add Wraith so that regressions can be easily spotted. Runs against a set of supported examples. Run while pointing government-frontend at the dummy content store, and an instance of static. Usage: On master run: wraith history test/wraith/config.yaml On local branch run: wraith latest test/wraith/config.yaml Had to run `bundle update` to get a version of nokogiri that would work with `wraith`. Configuration generated with `wraith setup` and combined with our existing wraith usage, [1]. With modern versions of wraith you don't need a `snap.js` file, as wraith will take care of the `phantomjs` intergration, however we still need to set some cookies to prevent intermitent UI (like the survey, or cookie bar) from interfering with the diff results. [1] alphagov/design-principles#246 This config seems to work OK-ish, though starts having time out issues with many combinations of screen_widths, dogpiling the rails app and getting timeouts. I couldn't find an obvious way to limit this without forking wraith, so for now i've used a smaller set of screen_widths. I also had to specifcy explicit heights for each screen_width, this seems to be new to Wraith 3.x. Previous versions would capture the full page as a default, where as now it will default to 1500px unless you specificy a height. This isn't ideal as most of the page on a small screen_size is much further down than 1500px. For now i've set some heights that deal with some common cases, but i'm not sure this is a good approach long term. There are some related wraith notes on that here: - bbc/wraith#318 (suggestion doesn't work) - bbc/wraith#296
- Loading branch information
David Singleton
committed
Feb 1, 2016
1 parent
ffec9eb
commit ecc15c1
Showing
5 changed files
with
160 additions
and
23 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| ############################################################## | ||
| ############################################################## | ||
| # This is an example configuration provided by Wraith. | ||
| # Feel free to amend for your own requirements. | ||
| # --- | ||
| # This particular config is intended to demonstrate how | ||
| # to use Wraith in 'history' mode, which is best suited to | ||
| # making sure your site's appearance remains consistent over | ||
| # time. | ||
| # | ||
| # `wraith history history.yaml` # generate base screenshots | ||
| # `wraith latest history.yaml` # take new shots and compare | ||
| # --- | ||
| # Installing: | ||
| # | ||
| # On a GOV.UK VM, run: | ||
| # `apt-get install imagemagick phantomjs` | ||
| # | ||
| # Otherwise, on a Mac, run: | ||
| # `brew install imagemagick phantomjs` | ||
| # | ||
| ############################################################## | ||
| ############################################################## | ||
|
|
||
| # (required) The engine to run Wraith with. Examples: 'phantomjs', 'casperjs', 'slimerjs' | ||
| browser: | ||
| phantomjs: 'phantomjs' | ||
|
|
||
| # (optional) JavaScript file to execute before taking screenshot of every path. Default: nil | ||
| before_capture: 'test/wraith/cookies_and_headers.js' | ||
|
|
||
| # (required) The directory that your latest screenshots will be stored in | ||
| directory: 'tmp/wraith/shots' | ||
|
|
||
| # (required for history mode, otherwise optional) The directory that your base screenshots will be stored in. | ||
| history_dir: 'tmp/wraith/shots_history' | ||
|
|
||
| # (required) Amount of fuzz ImageMagick will use when comparing images. A higher fuzz makes the comparison less strict. | ||
| fuzz: '20%' | ||
|
|
||
| # (required) The domain to take screenshots of. | ||
| domains: | ||
| dev: 'http://localhost:3090' | ||
|
|
||
| # (required) Screen widths (and optional height) to resize the browser to before taking the screenshot. | ||
| screen_widths: | ||
| - 320x5000 | ||
| - 600x4000 | ||
| - 1080x3000 | ||
|
|
||
| # (optional) Resize to each screen width (efficient), or reload at each screen width (costly). Default: 'reload' | ||
| resize_or_reload: 'reload' | ||
|
|
||
| # (optional) Color to highlight the image diff. Default: 'blue' | ||
| highlight_color: 'red' | ||
|
|
||
| # (optional) Choose which results are displayed in the gallery, and in what order. Default: alphanumeric | ||
| # Options: | ||
| # alphanumeric - all paths (with or without a difference) are shown, sorted by path | ||
| # diffs_first - all paths (with or without a difference) are shown, sorted by difference size (largest first) | ||
| # diffs_only - only paths with a difference are shown, sorted by difference size (largest first) | ||
| # Note: different screen widths are always grouped together. | ||
| mode: 'diffs_first' | ||
|
|
||
| # (optional) The maximum acceptable level of difference (in %) between two images before Wraith reports a failure. Default: 0 | ||
| threshold: 5 | ||
|
|
||
| # (required) The paths to capture. | ||
| paths: | ||
| case_study: '/government/case-studies/doing-business-in-spain.es' | ||
| case_study_withdrawn: '/government/case-studies/terence-age-33-stoke-on-trent' | ||
| take_part: '/government/get-involved/take-part/become-a-councillor' | ||
| statistics_announcement: '/government/statistics/announcements/diagnostic-imaging-dataset-for-september-2015--2' | ||
| statistics_national: '/government/statistics/announcements/uk-armed-forces-quarterly-personnel-report-october-2015' | ||
| statistics_announcement_release_date_changes: '/government/statistics/announcements/diagnostic-imaging-dataset-for-september-2015--1' | ||
| statistics_announcement_cancelled: '/government/statistics/announcements/diagnostic-imaging-dataset-for-september-2015' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| // ###################################################### | ||
| // This is an example module provided by Wraith. | ||
| // Feel free to amend for your own requirements. | ||
| // ###################################################### | ||
| module.exports = function (phantom, ready) { | ||
| phantom.addCookie({ | ||
| 'name': 'govuk_takenUserSatisfactionSurvey', | ||
| 'value': 'yes', | ||
| 'domain': 'localhost' | ||
| }); | ||
|
|
||
| phantom.addCookie({ | ||
| 'name': 'seen_cookie_message', | ||
| 'value': 'yes', | ||
| 'domain': 'localhost' | ||
| }); | ||
|
|
||
| phantom.open(phantom.url, function () { | ||
| setTimeout(ready, 1000); | ||
| }); | ||
| } |