Skip to content

Commit

Permalink
source-zendesk-support: checkpoint TicketMetricEvents state after e…
Browse files Browse the repository at this point in the history
…very 1000 documents

Previously, `TicketMetricEvents` was only emitting documents and
checkpointing state after reading all available documents from Zendesk
Support. `TicketMetricEvents` can take multiple hours to backfill, and
any task restarts during the backfill caused `TicketMetricEvents` to
start over completely instead of picking back up where it left off
before the restart.

Setting `state_checkpoint_interval` tells `TickeMetricEvents` to
checkpoint state & emit documents after every 1000 documents and allows
the stream to pick up where it left off if it's interrupted during a
backfill.
  • Loading branch information
Alex-Bair committed Oct 3, 2024
1 parent 5c263e1 commit 39fdec1
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions source-zendesk-support/source_zendesk_support/streams.py
Original file line number Diff line number Diff line change
Expand Up @@ -932,6 +932,7 @@ class TicketMetricEvents(SourceZendeskSupportCursorPaginationStream):
TicketMetricEvents stream: https://developer.zendesk.com/api-reference/ticketing/tickets/ticket_metric_events/
"""

state_checkpoint_interval = 1000
cursor_field = "time"
page_size = 1000

Expand Down

0 comments on commit 39fdec1

Please sign in to comment.