Screenshots for Localization

All the l10n screenshots are generated through Xcode UI Tests. (XCTestCase subclasses). These tests are being run as part of the L10nSnapshotTests scheme. They should work on any device but recently we've only run them on iPhone device.

The tests are under Client > Snapshot > L10nSnapshotTests in the project navigator. There is a L10nBaseSnapshotTests that has some shared test methods and then the tests are split up between L10nIntroSnapshotTests, L10nPermissionStringsSnapshotTests and L10nSnapshotTests.

Running the tests locally

There are two ways to run the tests. You can run them from the command line, which is done for automation, or you can run them in Xcode. The latter is the easiest for working on the tests since you can use recording and the debugger.

Running from the command line

If you want to run in different locales, first grab the l10n scripts repository:

cd firefox-ios
git clone [email protected]:mozilla-mobile/ios-l10n-scripts.git

Then import the locales into the application. Note that this modifies the project file and makes a ton of changes to the project. When you commit changes to a screenshot test, you have to be careful to not accidentally include changes from the import.

./ios-l10n-scripts/import-locales-firefox.sh

The automation is setup around Fastlane. It is recommended to pull in Ruby via Brew and then grab the gem for Fastlane. (This so that you do not need root or need to change the system provided Ruby). If you already installed fastlane, make sure to upgrade it, it changes often.

brew install ruby
gem install fastlane # Install or update fastlane

You can now generate screenshots as follows:

# Generate screenshots for all locales
./l10n-screenshots.sh

#Generate screenshots for just en-US and nl
./l10n-screenshots.sh en-US nl

The script is pretty silent. If it fails then you can look at l10n-screenshots/$LOCALE/snapshot.log. The log contains color escape codes and a lot of Emojis, like the Fastlane team prefers, so it may be more readable if you just cat it to your terminal.

If you want real-time feedback then just remove the log redirection temporarily from the l10n-screenshots.sh script so that it always displays what it is doing.

Running from Xcode locally

Running tests from Xcode is as simple as changing the current scheme to L10n Snapshot Tests and selecting an iPhone simulator. Then hit the little diamond next to a test and it will build and run in the selected simulator.

This is a normal test session so you can set breakpoints, look at the logs, etc.

Screenshots made with the snapshot() method are saved in $HOME/Library/Caches/tools.fastlane/screenshots,

You will need to create this directory before you start, otherwise you get lots of errors.

# Create the screenshots directory
mkdir -p ~/Library/Caches/tools.fastlane/screenshots

Then looking at them:

# Look at the screenshots you made from Xcode.
open ~/Library/Caches/tools.fastlane/screenshots -a preview

You can also go into the Report Navigator and see the screenshots that Xcode made as part of your test run.

Running the tests on CI

Thanks a solution that was developed integrating Taskcluster with Bitrise. L10n tests run on CI and the screenshots for all locales are taken and stored automatically.

The only manual part is to trigger the hook so that the tests start, but once that happens the rest is all automated. As a quick summary, the Taskcluster trigger generates a firefox-iOS build and runs the test in a locale to check the status. If the build works, then it splits the locales (~95) in groups of 6 per job and schedules the different jobs needed so that they run on Bitrise. Each Bitrsie job just run the l10n tests without building and sends the artifacts, with the locales in zip files to Taskcluster where they are stored.

The trigger to get the latest screenshots is still manual since it happens on a demand way after talking with the l10n team and looking for the best time to get the release screenshot. If that specific time could set and known in advance in the future, that step could be automated too, but so far the communication is great and the screenshots are generated on time for each release.

This is the static link that is updated with the latest release's screenshots

For more info contact Mobile Test Engineering team, slack : #mobile-testeng