Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wraps transaction into separate class #213

Merged
merged 5 commits into from
Jul 25, 2019
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
All notable changes to this project will be documented in this file.
This project adheres to [Semantic Versioning](http://semver.org/).

## [upcoming]

### Changed
* Separates read-only logic from logic that requires a transaction

## [4.1.0] - 2019-07-10
### Removed
* `--skip-check-package-names`. When pushing or checking an APK, expected package names must always be provided
Expand Down
21 changes: 10 additions & 11 deletions mozapkpublisher/check_rollout.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,17 @@
import requests

from argparse import ArgumentParser
from mozapkpublisher.common.googleplay import EditService, add_general_google_play_arguments

from mozapkpublisher.common.googleplay import add_general_google_play_arguments, \
GooglePlayConnection, GooglePlayEdit

DAY = 24 * 60 * 60

logger = logging.getLogger(__name__)


def check_rollout(edit_service, days):
def check_rollout(google_play, days):
"""Check if package_name has a release on staged rollout for too long"""
track_status = edit_service.get_track_status(track='production')
track_status = google_play.get_track_status(track='production')
releases = track_status['releases']
for release in releases:
if release['status'] == 'inProgress':
Expand All @@ -40,14 +40,13 @@ def main():
type=int, default=7)
config = parser.parse_args()

edit_service = EditService(
with GooglePlayEdit.transaction(GooglePlayConnection.open(
config.service_account,
config.google_play_credentials_file.name,
package_name='org.mozilla.firefox',
)
for (release, age) in check_rollout(edit_service, config.days):
print('fennec {} is on staged rollout at {}% but it shipped {} days ago'.format(
release['name'], int(release['userFraction'] * 100), int(age / DAY)))
config.google_play_credentials_file.name
), 'org.mozilla.firefox', True) as google_play:
for (release, age) in check_rollout(google_play, config.days):
print('fennec {} is on staged rollout at {}% but it shipped {} days ago'.format(
release['name'], int(release['userFraction'] * 100), int(age / DAY)))


__name__ == '__main__' and main()
7 changes: 0 additions & 7 deletions mozapkpublisher/common/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,6 @@ def __init__(self, checked_file, expected, actual):
)


class NoTransactionError(LoggedError):
def __init__(self, package_name):
super(NoTransactionError, self).__init__(
'Transaction has not been started for package "{}"'.format(package_name)
)


class NotMultiLocaleApk(LoggedError):
def __init__(self, apk_path, unique_locales):
super(NotMultiLocaleApk, self).__init__(
Expand Down
215 changes: 114 additions & 101 deletions mozapkpublisher/common/googleplay.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@
"""

import argparse
from contextlib import contextmanager

import httplib2
import json
import logging
Expand All @@ -23,7 +25,7 @@
# HACK: importing mock in production is useful for option `--do-not-contact-google-play`
from unittest.mock import MagicMock

from mozapkpublisher.common.exceptions import NoTransactionError, WrongArgumentGiven
from mozapkpublisher.common.exceptions import WrongArgumentGiven

logger = logging.getLogger(__name__)

Expand All @@ -41,57 +43,102 @@ def add_general_google_play_arguments(parser):
--service-account and --credentials must still be provided (you can just fill them with random string and file).''')


class EditService(object):
def __init__(self, service_account, credentials_file_path, package_name, commit=False, contact_google_play=True):
self._contact_google_play = contact_google_play
if self._contact_google_play:
general_service = _connect(service_account, credentials_file_path)
self._service = general_service.edits()
else:
self._service = _craft_google_play_service_mock()
logger.warning('`--do-not-contact-google-play` option was given. Not a single request to Google Play will be made!')
class GooglePlayConnection:
def __init__(self, edit_resource):
self._edit_resource = edit_resource

self._package_name = package_name
self._commit = commit
self.start_new_transaction()

def start_new_transaction(self):
result = self._service.insert(body={}, packageName=self._package_name).execute()
self._edit_id = result['id']

def transaction_required(method):
def _transaction_required(*args, **kwargs):
edit_service = args[0]
if edit_service._edit_id is None:
raise NoTransactionError(edit_service._package_name)

return method(*args, **kwargs)
return _transaction_required

@transaction_required
def commit_transaction(self):
if self._commit:
self._service.commit(editId=self._edit_id, packageName=self._package_name).execute()
logger.info('Changes committed')
logger.debug('edit_id "{}" for package "{}" has been committed'.format(self._edit_id, self._package_name))
else:
logger.warning('`commit` option was not given. Transaction not committed.')
def get_edit_resource(self):
return self._edit_resource

@staticmethod
def open(service_account, credentials_file_path):
# Create an httplib2.Http object to handle our HTTP requests an
# authorize it with the Credentials. Note that the first parameter,
# service_account_name, is the Email address created for the Service
# account. It must be the email address associated with the key that
# was created.
scope = 'https://www.googleapis.com/auth/androidpublisher'
credentials = ServiceAccountCredentials.from_p12_keyfile(
service_account,
credentials_file_path,
scopes=scope
)
http = httplib2.Http()
http = credentials.authorize(http)

service = build(serviceName='androidpublisher', version='v3', http=http,
cache_discovery=False)

return GooglePlayConnection(service.edits())


class _ExecuteDummy:
def __init__(self, return_value):
self._return_value = return_value

def execute(self):
return self._return_value


class MockGooglePlayConnection:
@staticmethod
def get_edit_resource():
edit_service_mock = MagicMock()

edit_service_mock.insert = lambda *args, **kwargs: _ExecuteDummy(
{'id': 'fake-transaction-id'})
edit_service_mock.commit = lambda *args, **kwargs: _ExecuteDummy(None)

apks_mock = MagicMock()
apks_mock.upload = lambda *args, **kwargs: _ExecuteDummy(
{'versionCode': 'fake-version-code'})
edit_service_mock.apks = lambda *args, **kwargs: apks_mock

update_mock = MagicMock()
update_mock.update = lambda *args, **kwargs: _ExecuteDummy('fake-update-response')
edit_service_mock.tracks = lambda *args, **kwargs: update_mock
edit_service_mock.listings = lambda *args, **kwargs: update_mock
edit_service_mock.apklistings = lambda *args, **kwargs: update_mock

return edit_service_mock


def connection_for_options(contact_google_play, service_account, credentials_file):
if contact_google_play:
return GooglePlayConnection.open(service_account, credentials_file.name)
else:
logger.warning('Not a single request to Google Play will be made, since `contact_google_play` was set')
return MockGooglePlayConnection()


class GooglePlayEdit:
"""Represents an "edit" to an app on the Google Play store

self._edit_id = None
Create an instance by calling GooglePlayEdit.transaction(), instead of using the
constructor. This can optionally handle committing the transaction when the "with" block
ends.

E.g.: `with GooglePlayEdit.transaction() as google_play:`
"""

def __init__(self, edit_resource, edit_id, package_name):
self._edit_resource = edit_resource
self._edit_id = edit_id
self._package_name = package_name

@transaction_required
def get_track_status(self, track):
response = self._service.tracks().get(
editId=self._edit_id, track=track, packageName=self._package_name
response = self._edit_resource.tracks().get(
editId=self._edit_id,
track=track,
packageName=self._package_name
).execute()
logger.debug(u'Track "{}" has status: {}'.format(track, response))
logger.debug('Track "{}" has status: {}'.format(track, response))
return response

@transaction_required
def upload_apk(self, apk_path):
logger.info('Uploading "{}" ...'.format(apk_path))
try:
response = self._service.apks().upload(
response = self._edit_resource.apks().upload(
editId=self._edit_id,
packageName=self._package_name,
media_body=apk_path
Expand All @@ -103,19 +150,17 @@ def upload_apk(self, apk_path):
# XXX This is really how data is returned by the googleapiclient.
error_content = json.loads(e.content)
errors = error_content['error']['errors']
if (
len(errors) == 1 and
errors[0]['reason'] in (
'apkUpgradeVersionConflict', 'apkNotificationMessageKeyUpgradeVersionConflict'
)
):
if (len(errors) == 1 and errors[0]['reason'] in (
'apkUpgradeVersionConflict',
'apkNotificationMessageKeyUpgradeVersionConflict'
)):
logger.warning(
'APK "{}" has already been uploaded on Google Play. Skipping...'.format(apk_path)
'APK "{}" has already been uploaded on Google Play. Skipping...'.format(
apk_path)
)
return
raise

@transaction_required
def update_track(self, track, version_codes, rollout_percentage=None):
body = {
u'releases': [{
Expand All @@ -125,32 +170,32 @@ def update_track(self, track, version_codes, rollout_percentage=None):
}
if rollout_percentage is not None:
if rollout_percentage < 0 or rollout_percentage > 100:
raise WrongArgumentGiven('rollout percentage must be between 0 and 100. Value given: {}'.format(rollout_percentage))
raise WrongArgumentGiven(
'rollout percentage must be between 0 and 100. Value given: {}'.format(
rollout_percentage))

body[u'userFraction'] = rollout_percentage / 100.0 # Ensure float in Python 2

response = self._service.tracks().update(
response = self._edit_resource.tracks().update(
editId=self._edit_id, track=track, packageName=self._package_name, body=body
).execute()
logger.info('Track "{}" updated with: {}'.format(track, body))
logger.debug('Track update response: {}'.format(response))

@transaction_required
def update_listings(self, language, title, full_description, short_description):
body = {
'fullDescription': full_description,
'shortDescription': short_description,
'title': title,
}
response = self._service.listings().update(
response = self._edit_resource.listings().update(
editId=self._edit_id, packageName=self._package_name, language=language, body=body
).execute()
logger.info(u'Listing for language "{}" has been updated with: {}'.format(language, body))
logger.debug(u'Listing response: {}'.format(response))

@transaction_required
def update_whats_new(self, language, apk_version_code, whats_new):
response = self._service.apklistings().update(
response = self._edit_resource.apklistings().update(
editId=self._edit_id, packageName=self._package_name, language=language,
apkVersionCode=apk_version_code, body={'recentChanges': whats_new}
).execute()
Expand All @@ -159,48 +204,16 @@ def update_whats_new(self, language, apk_version_code, whats_new):
))
logger.debug(u'Apk listing response: {}'.format(response))


def _craft_google_play_service_mock():
edit_service_mock = MagicMock()

edit_service_mock.insert = lambda *args, **kwargs: _ExecuteDummy({'id': 'fake-transaction-id'})
edit_service_mock.commit = lambda *args, **kwargs: _ExecuteDummy(None)

apks_mock = MagicMock()
apks_mock.upload = lambda *args, **kwargs: _ExecuteDummy({'versionCode': 'fake-version-code'})
edit_service_mock.apks = lambda *args, **kwargs: apks_mock

update_mock = MagicMock()
update_mock.update = lambda *args, **kwargs: _ExecuteDummy('fake-update-response')
edit_service_mock.tracks = lambda *args, **kwargs: update_mock
edit_service_mock.listings = lambda *args, **kwargs: update_mock
edit_service_mock.apklistings = lambda *args, **kwargs: update_mock

return edit_service_mock


class _ExecuteDummy():
def __init__(self, return_value):
self._return_value = return_value

def execute(self):
return self._return_value


def _connect(service_account, credentials_file_path):
""" Connect to the google play interface
"""

# Create an httplib2.Http object to handle our HTTP requests an
# authorize it with the Credentials. Note that the first parameter,
# service_account_name, is the Email address created for the Service
# account. It must be the email address associated with the key that
# was created.
scope = 'https://www.googleapis.com/auth/androidpublisher'
credentials = ServiceAccountCredentials.from_p12_keyfile(service_account, credentials_file_path, scopes=scope)
http = httplib2.Http()
http = credentials.authorize(http)

service = build(serviceName='androidpublisher', version='v3', http=http, cache_discovery=False)

return service
@staticmethod
@contextmanager
def transaction(connection, package_name, commit):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Default commit to False. This way the commit action is explicitly called out.

Context: If Google Play was more forgiving, I'd be okay to keep it as is, but reverting a change is impossible in some cases.

Copy link
Contributor Author

@mitchhentges mitchhentges Jul 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we leave it explicit? I definitely don't believe that we should default it to true, but I think that requiring devs to specify a value is a good thing. It reduces confusion when devs don't realize that the parameter is possible ("why aren't my changes being applied?"), but requires an explicit true if devs are confident that they want to apply their changes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to avoiding footguns, defaulting this param has another pro to me. commit becomes a kwarg, so if it's an incentive to call it this way:

with googleplay.transaction(connection, 'some.package.name', commit=True) as gp:
    pass

rather than that way:

with googleplay.transaction(connection, 'some.package.name', True) as gp:
    # What does True mean? 
    pass

While, I agree it doesn't prevent the latter from happening. It's just an incentive.

To me, the latter is more dangerous and can lead newcomers to just copy and paste an existing (and working) line while having very few context about what this True is about. I think the damage done by a committed transaction outweighs the damage done by the "why aren't my changes being applied" scenario. Moreover, the reason why changes aren't applied is explicitly called out in the logs. So guessing the fix shouldn't be a hard task.

Copy link
Contributor Author

@mitchhentges mitchhentges Jul 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I've found the best of both words: I've forced it to be a kwarg without giving it a default value:

def edit(..., package_name, *, commit):
    ...

Copy link
Contributor

@JohanLorenzo JohanLorenzo Jul 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oooooooh! TIL https://www.python.org/dev/peps/pep-3102/

That's an excellent finding 😃. Thanks!

edit_resource = connection.get_edit_resource()
edit_id = edit_resource.insert(body={}, packageName=package_name).execute()['id']
google_play = GooglePlayEdit(edit_resource, edit_id, package_name)
yield google_play
if commit:
edit_resource.commit(editId=edit_id, packageName=package_name)
logger.info('Changes committed')
logger.debug('edit_id "{}" for "{}" has been committed'.format(edit_id, package_name))
else:
logger.warning('Transaction not committed, since `commit` was `False`')
Loading