dadosfera · guilherme-maioli · Nov 30, 2021 · Nov 30, 2021 · Nov 30, 2021 · Nov 30, 2021
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -0,0 +1,15 @@
+# Description of change
+(write a short description here or paste a link to JIRA)
+
+# QA steps
+ - [ ] automated tests passing
+ - [ ] manual qa steps passing
+
+# Risks
+
+# Rollback steps
+ - revert this branch
+
+#### AI generated code
+https://internal.qlik.dev/general/ways-of-working/code-reviews/#guidelines-for-ai-generated-code
+- [ ] this PR has been written with the help of GitHub Copilot or another generative AI tool
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -0,0 +1,89 @@
+# Changelog
+
+## v1.8.0
+  * Updates API version to 17
+  * Updates pkg version to 25.0.0
+  * [#93](https://github.com/singer-io/tap-google-ads/pull/93)
+
+## v1.7.0
+  * Run on python 3.11.7 [#88](https://github.com/singer-io/tap-google-ads/pull/88)
+
+## v1.6.0
+  * Updates API version to 15
+  * Updates pkg version to 22.1.0
+  * [#86](https://github.com/singer-io/tap-google-ads/pull/86)
+
+## v1.5.0
+  * Updates API version to 13
+  * Updates pkg version to 21.0.0
+  * [#82](https://github.com/singer-io/tap-google-ads/pull/82)
+
+
+## v1.4.0
+  * Updates API version to 12
+  * Updates pkg version to 19.0.0
+  * Removes `gmail_ad` fields from `ad_performance_report` as they are no longer available after API version bump.
+  * [#76](https://github.com/singer-io/tap-google-ads/pull/76)
+
+## v1.3.4
+  * Updates API Version to 11
+  * Updates pkg version to 17.0.0
+  * [#79](https://github.com/singer-io/tap-google-ads/pull/79)
+
+## v1.3.3
+  * Update applicable core streams to use limit clause. Updates tests [#68](https://github.com/singer-io/tap-google-ads/pull/68)
+
+## v1.3.2
+  * Add timeout parameter to Google Ads search requests
+  * Allow for request_timeout config parameter to be provided [#64](https://github.com/singer-io/tap-google-ads/pull/64)
+
+## v1.3.1
+  * Handle uncaught exceptions [#61](https://github.com/singer-io/tap-google-ads/pull/61)
+  * Implement interruptible full table streams [#60](https://github.com/singer-io/tap-google-ads/pull/60)
+
+## v1.3.0 [#58](https://github.com/singer-io/tap-google-ads/pull/58)
+  * Adds several new core streams including ad_group_criterion, campaign_criterion, and their attributed resources.
+  * Adds new subclass UserInterestStream to handle stream specific name transformations.
+  * Renames obj and corresponding variables in all transform_keys functions.
+
+## v1.2.0
+  * Renames `REPORTS` variable to `STREAMS` and updates corresponding variables similarly. Removes unused `add_extra_fields` function [#56](https://github.com/singer-io/tap-google-ads/pull/56) 
+  * Adds `automatic_keys` to metadata for streams, including reports. Updates tests [#55](https://github.com/singer-io/tap-google-ads/pull/55)
+
+## v1.1.0
+  * Fixes a bug with currently_syncing and adds tests around the bug fix [#54](https://github.com/singer-io/tap-google-ads/pull/54)
+  * Adds `campaign_labels` and `labels` core streams; adds "campaign.labels" field to reports where relevant [#53](https://github.com/singer-io/tap-google-ads/pull/53)
+  * Adds `call_details` core stream and removes segmenting resources from core streams [#49](https://github.com/singer-io/tap-google-ads/pull/49)
+
+## v1.0.0
+  * Version bump for GA release
+  * Adds fields to click_view report definition [#44](https://github.com/singer-io/tap-google-ads/pull/44)
+  * Adds date ranges to tests for faster test runs [#43](https://github.com/singer-io/tap-google-ads/pull/43)
+  * Adds more tests around primary key hashing [#42](https://github.com/singer-io/tap-google-ads/pull/42)
+
+## v0.3.0
+  * Removes unused code
+  * Adds a behavior to "_sdc_record_hash"
+  * Fixed field exclusion for segments and attributes
+  * Adds tests for field exclusion
+  * Updates the "type_" field to "type" Transform type_ to type [#36](https://github.com/singer-io/tap-google-ads/pull/36)
+  * Updates fields in the report streams to include the Google Ads resource name Report streams prefix resource names [#37](https://github.com/singer-io/tap-google-ads/pull/37)
+  * Updates the generate_hash function to be explicit about the order of the fields getting hashed Change _sdc_record_hash to sorted list of tuples [#38](https://github.com/singer-io/tap-google-ads/pull/38)
+
+## v0.2.0 [#31](https://github.com/singer-io/tap-google-ads/pull/31)
+  * Add ability for the tap to use `currently_syncing` [#24](https://github.com/singer-io/tap-google-ads/pull/24)
+  * Add `end_date` as a configurable property to end a sync at a certain date [#28](https://github.com/singer-io/tap-google-ads/pull/28)
+  * Fix a field exculsion bug introduced in `v0.1.0` around metric compatibility [#29](https://github.com/singer-io/tap-google-ads/pull/29)
+
+## v0.1.0 [#23](https://github.com/singer-io/tap-google-ads/pull/23)
+  * Update bookmarks to only be written with a midnight time value
+  * Fix error logging to more concise
+  * Add retry logic to sync queries [#22](https://github.com/singer-io/tap-google-ads/pull/22)
+  * Fix field exclusion bug: Metrics can exclude attributes
+  * Rename:
+    * `adgroup_performance_report` to `ad_group_performance_report`
+    * `audience_performance_report` to `ad_group_audience_performance_report`
+  * Add `campaign_audience_performance_report`
+
+## v0.0.1
+  * Alpha Release [#13](https://github.com/singer-io/tap-google-ads/pull/13)
diff --git a/LICENSE b/LICENSE
@@ -7,4 +7,4 @@ This software and related documentation are provided under a license agreement c
 
 Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.
 
-If you have any questions about this, please contact Dadosfera.ai at [email protected]
+If you have any questions about this, please contact Dadosfera.ai at [email protected]
diff --git a/README.md b/README.md
@@ -1,123 +1,69 @@
-# ` tap-googleads` ![Build and Tests](https://github.com/AutoIDM/tap-googleads/actions/workflows/ci.yml/badge.svg?branch=main) [![PyPI download month](https://img.shields.io/pypi/dm/tap-googleads.svg)](https://pypi.python.org/pypi/tap-googleads/) 
-
-THIS IS NOT READY FOR PRODUCTION. Bearer tokens sometimes slip out to logs. Use at your own Peril :D
-
-## unique tap-googleads functionality
-1. `_sdc_primary_key` is added to each stream in order to give a primary_key because google's api has nested data that doesn't play nicely without a top level primary key, so we copy the data needed for a primary key to this made up field. All other fields match the api response.   
-
-
-## Capabilities
-
-* `catalog`
-* `discover`
-* `about`
-* `stream-maps`
-
-## Settings
-
-| Setting          | Required | Default | Description |
-|:-----------------|:--------:|:-------:|:------------|
-| client_id        | True     | None    | ClientID from Oauth Setup |
-| client_secret    | True     | None    | ClientSecret from Oauth Setup |
-| developer_token  | True     | None    | Developer Token from Google Ads Console |
-| refresh_token    | True     | None    | Refresh Token from Oauth dance |
-| customer_id      | True     | None    | Customer ID from Google Ads Console, note this should be the top level client. This tap will pull all subaccounts |
-| login_customer_id| True     | None    | Customer ID that has access to the customer_id, note that they can be the same, but they don't have to be as this could be a Manager account |
-| start_date       | True     | 2022-03-24T00:00:00Z (Today-7d) | Date to start our search from, applies to Streams where there is a filter date. Note that Google responds to Data in buckets of 1 Day increments |
-| end_date         | True     | 2022-03-31T00:00:00Z (Today) | Date to end our search on, applies to Streams where there is a filter date. Note that the query is BETWEEN start_date AND end_date |
-
-### Get refresh token
-1. GET https://accounts.google.com/o/oauth2/v2/auth?response_type=code&client_id=client_id&redirect_uri=http://127.0.0.1&scope=https://www.googleapis.com/auth/adwords&state=autoidm&access_type=offline&prompt=select_account&include_granted_scopes=true
-1. POST https://www.googleapis.com/oauth2/v4/token?code={code}&client_id={client_id}&client_secret={client_secret}&redirect_uri=http://127.0.0.1&grant_type=authorization_code
-1. POST https://www.googleapis.com/oauth2/v4/token?refresh_token={refres_token}&client_id={client_id}&client_secret={client_secret]&grant_type=refresh_token
-
-
-## Installation
-
-```bash
-pipx install tap-googleads
-```
+# tap_google_ads
+
+This is a [Singer](https://singer.io) tap that produces JSON-formatted
+data from the Google Ads API following the [Singer
+spec](https://github.com/singer-io/getting-started/blob/master/SPEC.md).
+
+This tap: 
+
+- Pulls data from the [Google Ads API](https://developers.google.com/google-ads/api/docs/start).
+- Extracts the following resources from Google Ads
+  - [Accessible Bidding Strategies](https://developers.google.com/google-ads/api/reference/rpc/v10/AccessibleBiddingStrategy)
+  - [Accounts](https://developers.google.com/google-ads/api/reference/rpc/v10/Customer)
+  - [Ad Groups](https://developers.google.com/google-ads/api/reference/rpc/v10/AdGroup)
+  - [Ads](https://developers.google.com/google-ads/api/reference/rpc/v10/Ad)
+  - [Bidding Strategies](https://developers.google.com/google-ads/api/reference/rpc/v10/BiddingStrategy)
+  - [Call Details](https://developers.google.com/google-ads/api/reference/rpc/v10/CallView)
+  - [Campaigns](https://developers.google.com/google-ads/api/reference/rpc/v10/Campaign)
+  - [Campaign Budgets](https://developers.google.com/google-ads/api/reference/rpc/v10/CampaignBudget)
+  - [Campaign Labels](https://developers.google.com/google-ads/api/reference/rpc/v10/CampaignLabel)
+  - [Labels](https://developers.google.com/google-ads/api/reference/rpc/v10/Label)
+  - [Reporting](https://developers.google.com/google-ads/api/docs/reporting/overview)
+    - [Account Performance Report](https://developers.google.com/google-ads/api/fields/v10/customer)
+    - [Ad Group Performance Report](https://developers.google.com/google-ads/api/fields/v10/ad_group)
+    - [Ad Group Audience Performance Report](https://developers.google.com/google-ads/api/fields/v10/ad_group_audience_view)
+    - [Ad Performance Report](https://developers.google.com/google-ads/api/fields/v10/ad_group_ad)
+    - [Age Range Performance Report](https://developers.google.com/google-ads/api/fields/v10/age_range_view)
+    - [Campaign Performance Report](https://developers.google.com/google-ads/api/fields/v10/campaign)
+    - [Campaign Audience Performance Report](https://developers.google.com/google-ads/api/fields/v10/campaign_audience_view)
+    - [Call Metrics Call Details Report](https://developers.google.com/google-ads/api/fields/v10/call_view)
+    - [Click Performance Report](https://developers.google.com/google-ads/api/fields/v10/click_view)
+    - [Display Keyword Performance Report](https://developers.google.com/google-ads/api/fields/v10/display_keyword_view)
+    - [Display Topics Performance Report](https://developers.google.com/google-ads/api/fields/v10/topic_view)
+    - [Expanded Landing Page Report](https://developers.google.com/google-ads/api/fields/v10/expanded_landing_page_view)
+    - [Gender Performance Report](https://developers.google.com/google-ads/api/fields/v10/gender_view)
+    - [Geo Performance Report](https://developers.google.com/google-ads/api/fields/v10/geographic_view)
+    - [Keywordless Query Report](https://developers.google.com/google-ads/api/fields/v10/dynamic_search_ads_search_term_view)
+    - [Keywords Performance Report](https://developers.google.com/google-ads/api/fields/v10/keyword_view)
+    - [Landing Page Report](https://developers.google.com/google-ads/api/fields/v10/landing_page_view)
+    - [Placeholder Feed Item Report](https://developers.google.com/google-ads/api/fields/v10/feed_item)
+    - [Placeholder Report](https://developers.google.com/google-ads/api/fields/v10/feed_placeholder_view)
+    - [Placement Performance Report](https://developers.google.com/google-ads/api/fields/v10/managed_placement_view)
+    - [Search Query Performance Report](https://developers.google.com/google-ads/api/fields/v10/search_term_view)
+    - [Shopping Performance Report](https://developers.google.com/google-ads/api/fields/v10/shopping_performance_view)
+    - [User Location Performance Report](https://developers.google.com/google-ads/api/fields/v10/user_location_view)
+    - [UserLocation Performance Report](https://developers.google.com/google-ads/api/fields/v10/user_location_view)
+    - [Video Performance Report](https://developers.google.com/google-ads/api/fields/v10/video)
+
+## Bookmarking Strategy
+
+The Google Ads API supports the `start_date` and `end_date` parameters that limits the records which filters the analytics records in the given time period.
 
 ## Configuration
 
-### Accepted Config Options
-
-
-A full list of supported settings and capabilities for this
-tap is available by running:
-
-```bash
-tap-googleads --about
-```
-
-## Usage
-
-You can easily run `tap-googleads` by itself or in a pipeline using [Meltano](https://meltano.com/).
-
-### Executing the Tap Directly
-
-```bash
-tap-googleads --version
-tap-googleads --help
-tap-googleads --config CONFIG --discover > ./catalog.json
-```
-
-## Developer Resources
-
-- [ ] `Developer TODO:` As a first step, scan the entire project for the text "`TODO:`" and complete any recommended steps, deleting the "TODO" references once completed.
-
-### Initialize your Development Environment
-
-```bash
-pipx install poetry
-poetry install
-```
-
-### Create and Run Tests
-
-Create tests within the `tap_googleads/tests` subfolder and
-  then run:
-
-```bash
-poetry run pytest
-```
+This tap requires a `config.json` which specifies details regarding [OAuth 2.0](https://developers.google.com/google-ads/api/docs/oauth/overview) authentication and a cutoff date for syncing historical data. See [config.sample.json](config.sample.json) for an example.
 
-You can also test the `tap-googleads` CLI interface directly using `poetry run`:
+To run the discover mode of `tap-google-ads` with the configuration file, use this command:
 
 ```bash
-poetry run tap-googleads --help
+$ tap-google-ads -c my-config.json -d
 ```
 
-### Testing with [Meltano](https://www.meltano.com)
-
-_**Note:** This tap will work in any Singer environment and does not require Meltano.
-Examples here are for convenience and to streamline end-to-end orchestration scenarios._
-
-Your project comes with a custom `meltano.yml` project file already created. Open the `meltano.yml` and follow any _"TODO"_ items listed in
-the file.
-
-Next, install Meltano (if you haven't already) and any needed plugins:
+To run the sync mode of `tap-google-ads` with the catalog file, use the command:
 
 ```bash
-# Install meltano
-pipx install meltano
-# Initialize meltano within this directory
-cd tap-googleads
-meltano install
+$ tap-google-ads -c my-config.json --catalog catalog.json
 ```
+---
 
-Now you can test and orchestrate using Meltano:
-
-```bash
-# Test invocation:
-meltano invoke tap-googleads --version
-# OR run a test `elt` pipeline:
-meltano elt tap-googleads target-jsonl
-```
-
-### SDK Dev Guide
-
-See the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to 
-develop your own taps and targets.
-
-Built with the [Meltano SDK](https://sdk.meltano.com) for Singer Taps and Targets.
+Copyright &copy; 2021 Stitch
diff --git a/config.sample.json b/config.sample.json
@@ -0,0 +1,8 @@
+{
+    "start_date": "2020-10-01T00:00:00Z",
+    "login_customer_ids": [{"customerId": "1234567890", "loginCustomerId": "0987654321"}],
+    "oauth_client_id":"client_id",
+    "oauth_client_secret":"client_secret",
+    "refresh_token":"refresh_token",
+    "developer_token":"developer_token"
+}
diff --git a/setup.py b/setup.py
@@ -0,0 +1,38 @@
+#!/usr/bin/env python
+
+from setuptools import setup
+
+setup(name='tap-google-ads',
+      version='1.8.0',
+      description='Singer.io tap for extracting data from the Google Ads API',
+      author='Stitch',
+      url='http://singer.io',
+      classifiers=['Programming Language :: Python :: 3 :: Only'],
+      py_modules=['tap_google_ads'],
+      install_requires=[
+          'singer-python==6.0.0',
+          'requests==2.26.0',
+          'backoff==2.2.1',
+          'google-ads==25.0.0',
+          'protobuf==5.28.0',
+
+          # Necessary to handle gRPC exceptions properly, documented
+          # in an issue here: https://github.com/googleapis/python-api-core/issues/301
+          'grpcio-status==1.66.1',
+      ],
+      extras_require= {
+          'dev': [
+              'pylint',
+              'nose',
+              'ipdb',
+          ]
+      },
+      entry_points='''
+          [console_scripts]
+          tap-google-ads=tap_google_ads:main
+      ''',
+      packages=['tap_google_ads'],
+      package_data = {
+      },
+      include_package_data=True,
+)
Original file line number	Diff line number	Diff line change
Expand Up		@@ -7,4 +7,4 @@ This software and related documentation are provided under a license agreement c

		Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

		If you have any questions about this, please contact Dadosfera.ai at [email protected]
		If you have any questions about this, please contact Dadosfera.ai at [email protected]