Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HGI-6418 Create data validation for the three required information #22

Merged
merged 1 commit into from
Sep 5, 2024

Conversation

butkeraites-hotglue
Copy link
Contributor

@butkeraites-hotglue butkeraites-hotglue commented Aug 30, 2024

Included data validation for the three required fields:

  • Name
  • StageName
  • CloseDate

If the data is not present a message is included in target_state.json

f'StageName (pipeline_stage_id: {record.get("pipeline_stage_id")} or status: {record.get("status")}), '
f'CloseDate (close_date: {has_close_date})'
)
except Exception as exc:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to put the try except on all the preprocess_record logic, because if any case that we didn't consider fails it will stop the job and the state won't be generated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@@ -391,23 +391,39 @@ class DealsSink(SalesforceV3Sink):
def reference_data(self):
params = {"q": "SELECT id, name from Account"}
response = self.request_api("GET", endpoint="query", params=params)
response = response.json()["records"]
response = response.json().get("records",{})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if records is not present I think it should default to a list so next line doesn't fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@butkeraites-hotglue butkeraites-hotglue force-pushed the HGI-6418-fix-key-errors-deal-sink branch from 49e174b to 73ce384 Compare August 30, 2024 18:56
@butkeraites-hotglue butkeraites-hotglue force-pushed the HGI-6418-fix-key-errors-deal-sink branch from 73ce384 to 23e8082 Compare August 30, 2024 19:01
@hsyyid hsyyid merged commit 9f586fb into main Sep 5, 2024
0 of 5 checks passed
@hsyyid hsyyid deleted the HGI-6418-fix-key-errors-deal-sink branch September 5, 2024 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants