Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

metadata setting to FULL_TABLE doesn't do a full_sync it seems to always use incremental #126

Open
visch opened this issue Oct 24, 2024 · 0 comments

Comments

@visch
Copy link
Member

visch commented Oct 24, 2024

Maybe I've done something wrong

  - name: tap-universal-file
    namespace: tap_universal_file
    pip_url: tap-universal-file[s3]@git+https://github.com/MeltanoLabs/tap-universal-file.git
    capabilities:
    - state
    - catalog
    - discover
    - about
    - stream-maps
    settings:
    - name: stream_name
    - name: protocol
    - name: file_path
    - name: file_regex
    - name: file_type
    - name: fail_when_no_files_found
      kind: boolean
    - name: compression
    - name: additional_info
      kind: boolean
    - name: start_date
      kind: date_iso8601
    - name: delimited_error_handling
    - name: delimited_delimiter
    - name: delimited_quote_character
    - name: delimited_header_skip
      kind: integer
    - name: delimited_footer_skip
      kind: integer
    - name: delimited_override_headers
      kind: array
    - name: jsonl_error_handling
    - name: jsonl_sampling_strategy
    - name: jsonl_type_coercion_strategy
    - name: avro_type_coercion_strategy
    - name: parquet_type_coercion_strategy
    - name: s3_anonymous_connection
    - name: AWS_ACCESS_KEY_ID
      kind: string
      sensitive: true
    - name: AWS_SECRET_ACCESS_KEY
      kind: string
      sensitive: true
    - name: caching_strategy
    metadata:
      '*':
        replication-method: FULL_TABLE

With this set I keep getting incremental syncs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant