Bug 1560713 - Move to Google API v3 #207

JohanLorenzo · 2019-06-24T15:24:41Z

Fixed. It was a bug on the Google Play server. ~~Please don't review yet, the code is currently buggy. It returns:~~

2019-06-24 17:21:17,203 - extractor.py - INFO - Extracting metadata from a copy of "tmp/target-1.apk"...
2019-06-24 17:21:17,516 - extractor.py - INFO - Extracting metadata from a copy of "tmp/target-3.apk"...
2019-06-24 17:21:17,788 - extractor.py - INFO - Extracting metadata from a copy of "tmp/target-4.apk"...
2019-06-24 17:21:18,012 - extractor.py - INFO - Extracting metadata from a copy of "tmp/target.apk"...
2019-06-24 17:21:18,220 - checker.py - INFO - Checking APKs' metadata and content...
2019-06-24 17:21:18,221 - checker.py - INFO - Expected product types ['org.mozilla.fennec_aurora'] found
2019-06-24 17:21:18,221 - checker.py - INFO - Firefox version "68.0a1" matches package name "org.mozilla.fennec_aurora"
2019-06-24 17:21:18,221 - checker.py - INFO - All APKs have the same Firefox version: 68.0a1
2019-06-24 17:21:18,221 - checker.py - INFO - All APKs have the same Firefox BuildID: 20190621074959
2019-06-24 17:21:18,221 - history.py - DEBUG - Expected to find these combos for Firefox 68.0a1: x86 API 16+, arm64-v8a API 21+, x86_64 API 21+, armeabi-v7a API 16+
2019-06-24 17:21:18,221 - checker.py - INFO - Every expected APK was found!
2019-06-24 17:21:18,221 - checker.py - INFO - "tmp/target-1.apk" is multilocale.
2019-06-24 17:21:18,221 - checker.py - INFO - "tmp/target-3.apk" is multilocale.
2019-06-24 17:21:18,221 - checker.py - INFO - "tmp/target-4.apk" is multilocale.
2019-06-24 17:21:18,221 - checker.py - INFO - "tmp/target.apk" is multilocale.
2019-06-24 17:21:18,221 - checker.py - INFO - All APKs have the same locales: ('an', 'ar', 'ast', 'az', 'be', 'bg', 'bn', 'br', 'bs', 'ca', 'cak', 'cs', 'cy', 'da', 'de', 'dsb', 'el', 'en-CA', 'en-GB', 'en-US', 'eo', 'es-AR', 'es-CL', 'es-ES', 'es-MX', 'et', 'eu', 'fa', 'ff', 'fi', 'fr', 'fy-NL', 'ga-IE', 'gd', 'gl', 'gn', 'gu-IN', 'he', 'hi-IN', 'hr', 'hsb', 'hu', 'hy-AM', 'id', 'is', 'it', 'ja', 'ka', 'kab', 'kk', 'kn', 'ko', 'lij', 'lo', 'lt', 'lv', 'ml', 'mr', 'ms', 'my', 'nb-NO', 'ne-NP', 'nl', 'nn-NO', 'oc', 'pa-IN', 'pl', 'pt-BR', 'pt-PT', 'rm', 'ro', 'ru', 'sk', 'sl', 'son', 'sq', 'sr', 'sv-SE', 'ta', 'te', 'th', 'tr', 'trs', 'uk', 'ur', 'uz', 'vi', 'wo', 'xh', 'zam', 'zh-CN', 'zh-TW')
2019-06-24 17:21:18,221 - checker.py - INFO - APKs version codes are correctly ordered: {'2015635775': 'x86_64', '2015635773': 'x86', '2015635771': 'arm64-v8a', '2015635769': 'armeabi-v7a'}
2019-06-24 17:21:18,221 - checker.py - INFO - APKs are sane!
2019-06-24 17:21:18,225 - discovery.py - INFO - URL being requested: GET https://www.googleapis.com/discovery/v1/apis/androidpublisher/v3/rest
2019-06-24 17:21:18,225 - transport.py - INFO - Attempting refresh to obtain initial access_token
2019-06-24 17:21:18,228 - crypt.py - DEBUG - [b'eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9', b'eyJhdWQiOiJodHRwczovL29hdXRoMi5nb29nbGVhcGlzLmNvbS90b2tlbiIsInNjb3BlIjoiaHR0cHM6Ly93d3cuZ29vZ2xlYXBpcy5jb20vYXV0aC9hbmRyb2lkcHVibGlzaGVyIiwiaWF0IjoxNTYxMzg5Njc4LCJleHAiOjE1NjEzOTMyNzgsImlzcyI6ImpvaGFuLWxvcmVuem8tc2VydmljZS1hY2NvdW50QGJveHdvb2QtYXhvbi04MjUuaWFtLmdzZXJ2aWNlYWNjb3VudC5jb20ifQ', b'GQeXkwaCQYtEcCyEWEjAULC7qJrLYuG0SBMtQCCKBOOvkyN91-qBa3gc13YH0mxyGsAlK__i6vnvmdLuPS0K8whXgjDxFut_9p-yby6Y-HaMX3DpwAu0v6q84I042adsvN-mxWov5UtD3o-xhXMpLyKbNzs6CMVaiLT1adyldWllSWsDzEggysPdDLW2W876wKoD9junOT52nF0A9ONz1jN24jQ1ZZ9wX6__YO-8B6J6yQKl9puJ5n7KstcQTxXbHP-WZml-twEixfzAViJ43fRRFe6OmA4gWiVT4sf2nRmXRxtcukkUZZrUcRjy_vmbSACQNO6JjRa9jz2gawyjRA']
2019-06-24 17:21:18,228 - client.py - INFO - Refreshing access_token
2019-06-24 17:21:18,404 - discovery.py - INFO - URL being requested: POST https://www.googleapis.com/androidpublisher/v3/applications/org.mozilla.fennec_aurora/edits?alt=json
2019-06-24 17:21:19,620 - googleplay.py - INFO - Uploading "tmp/target-1.apk"...
2019-06-24 17:21:19,621 - discovery.py - WARNING - media_mime_type argument not specified: trying to auto-detect for tmp/target-1.apk
2019-06-24 17:21:19,634 - discovery.py - INFO - URL being requested: POST https://www.googleapis.com/upload/androidpublisher/v3/applications/org.mozilla.fennec_aurora/edits/02121994255104624805/apks?alt=json&uploadType=media
Traceback (most recent call last):
  File "./mozapkpublisher/push_apk.py", line 231, in <module>
    __name__ == '__main__' and main()
  File "./mozapkpublisher/push_apk.py", line 225, in main
    config.skip_checks_fennec
  File "./mozapkpublisher/push_apk.py", line 111, in push_apk
    rollout_percentage,
  File "./mozapkpublisher/push_apk.py", line 135, in _upload_apks
    edit_service.upload_apk(path)
  File "/Users/johanlorenzo/git/mozilla-releng/mozapkpublisher/mozapkpublisher/common/googleplay.py", line 68, in _transaction_required
    return method(*args, **kwargs)
  File "/Users/johanlorenzo/git/mozilla-releng/mozapkpublisher/mozapkpublisher/common/googleplay.py", line 97, in upload_apk
    media_body=apk_path
  File "/Users/johanlorenzo/.virtualenvs/mozapkpublisher/lib/python3.7/site-packages/google_api_python_client-1.7.8-py3.7.egg/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/Users/johanlorenzo/.virtualenvs/mozapkpublisher/lib/python3.7/site-packages/google_api_python_client-1.7.8-py3.7.egg/googleapiclient/http.py", line 851, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 500 when requesting https://www.googleapis.com/upload/androidpublisher/v3/applications/org.mozilla.fennec_aurora/edits/02121994255104624805/apks?alt=json&uploadType=media returned "">

~~I contacted the Google Play support.~~

Fixes #67

coveralls · 2019-06-24T15:27:59Z

Coverage increased (+0.2%) to 94.79% when pulling 839631d on JohanLorenzo:bug-1560713 into dfd757f on mozilla-releng:master.

mitchhentges · 2019-07-05T00:03:01Z

mozapkpublisher/common/googleplay.py

@@ -79,9 +79,17 @@ def commit_transaction(self):

        self._edit_id = None

+    @transaction_required


This comment doesn't block or even relate to this PR, it's a more general mozapkpublisher musing:

I wonder if, instead of having a mutable EditService and having checks before operation to assert state (which is what we're doing relatively ergonomically with this @transaction_required annotation), we could utilize typestates. So, you'll call a function to start the transaction, and you'll receive a transaction object that can perform all the operations (and commit at the end).

This way, it's easier to use this API, since you don't have functions on your objects that have implicit requirements - you physically can't call transaction functions until you've started the transaction. Additionally, there's less room for errors when changing the API, such as forgetting the @transaction_required annotation, since required resources (like edit_id) would sit in the transaction object.

I'm tempted to create a future work ticket, but I'm interested in your thoughts :)

That's a great idea! Yeah, we should get rid of this @transaction_required pattern, we don't even create multiple transactions in the code base. That's a YAGNI I could have avoided.

I've never heard of the typestates pattern. After glancing at it, it seems this pattern can be achieved by a context manager in python https://docs.python.org/3/library/contextlib.html. How does it sound to you?

I was originally focused on improving our types (splitting the "transaction" into its own type), but leveraging a context manager for commiting the transaction definitely improves ergonomics further, good call 😁

mozapkpublisher/common/googleplay.py

mitchhentges · 2019-07-05T00:07:44Z

mozapkpublisher/common/googleplay.py

@@ -188,6 +201,6 @@ def connect(service_account, credentials_file_path, api_version='v2'):
    http = httplib2.Http()
    http = credentials.authorize(http)

-    service = build('androidpublisher', api_version, http=http, cache_discovery=False)
+    service = build(serviceName='androidpublisher', version='v3', http=http, cache_discovery=False)


Thanks for hard-coding the api version here, that's 💯

JohanLorenzo force-pushed the bug-1560713 branch from e023c09 to 108f590 Compare July 3, 2019 16:00

JohanLorenzo changed the title ~~Bug 1560713 - WIP while 500 error is debugged~~ Bug 1560713 - Move to Google API v3 Jul 4, 2019

JohanLorenzo force-pushed the bug-1560713 branch 2 times, most recently from f15ac8b to ebd47bd Compare July 4, 2019 16:00

JohanLorenzo requested a review from mitchhentges July 4, 2019 16:10

JohanLorenzo force-pushed the bug-1560713 branch from ebd47bd to 4ab99e2 Compare July 4, 2019 16:33

mitchhentges approved these changes Jul 5, 2019

View reviewed changes

Bug 1560713 - Move to Google API v3

839631d

JohanLorenzo force-pushed the bug-1560713 branch from 4ab99e2 to 839631d Compare July 5, 2019 16:26

JohanLorenzo merged commit 0b1d978 into mozilla-releng:master Jul 5, 2019

JohanLorenzo deleted the bug-1560713 branch July 5, 2019 16:35

JohanLorenzo mentioned this pull request Jul 23, 2019

Wraps transaction into separate class #213

Merged

mitchhentges mentioned this pull request Aug 16, 2019

Revert to android-publisher API v2 #223

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug 1560713 - Move to Google API v3 #207

Bug 1560713 - Move to Google API v3 #207

JohanLorenzo commented Jun 24, 2019 •

edited

Loading

coveralls commented Jun 24, 2019 •

edited

Loading

mitchhentges Jul 5, 2019

JohanLorenzo Jul 5, 2019

mitchhentges Jul 5, 2019

mitchhentges Jul 5, 2019

		@@ -79,9 +79,17 @@ def commit_transaction(self):

		self._edit_id = None

		@transaction_required

Bug 1560713 - Move to Google API v3 #207

Bug 1560713 - Move to Google API v3 #207

Conversation

JohanLorenzo commented Jun 24, 2019 • edited Loading

coveralls commented Jun 24, 2019 • edited Loading

mitchhentges Jul 5, 2019

Choose a reason for hiding this comment

JohanLorenzo Jul 5, 2019

Choose a reason for hiding this comment

mitchhentges Jul 5, 2019

Choose a reason for hiding this comment

mitchhentges Jul 5, 2019

Choose a reason for hiding this comment

JohanLorenzo commented Jun 24, 2019 •

edited

Loading

coveralls commented Jun 24, 2019 •

edited

Loading