Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add more optional configs to tap-csv #891

Merged
merged 1 commit into from
Oct 12, 2022
Merged

add more optional configs to tap-csv #891

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion _data/meltano/extractors/tap-csv/meltanolabs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,20 @@ settings:
* `encoding`: [Optional] The file encoding to use when reading the file (i.e. "latin1", "UTF-8"). Use this setting when you get a UnicodeDecodeError error.
Each input CSV file must be a traditionally-delimited CSV (comma separated columns, newlines indicate new rows, double quoted values).

The following entries are passed through in an internal CSV dialect that then is used to configure the CSV reader:

* `delimiter`: A one-character string used to separate fields. It defaults to ','.
* `doublequote`: Controls how instances of quotechar appearing inside a field should themselves be quoted. When True, the character is doubled. When False, the escapechar is used as a prefix to the quotechar. It defaults to True.
* `escapechar`: A one-character string used by the reader, where the escapechar removes any special meaning from the following character. It defaults to None, which disables escaping.
* `quotechar`: A one-character string used to quote fields containing special characters, such as the delimiter or quotechar, or which contain new-line characters. It defaults to '"'.
* `skipinitialspace`: When True, spaces immediately following the delimiter are ignored. The default is False.
* `strict`: When True, raise exception Error on bad CSV input. The default is False.

The first row is the header defining the attribute name for that column and will result to a column of the same name in the database. It must have a valid format with no spaces or special characters (like for example `!` or `@`, etc).
- name: csv_files_definition
label: CSV Files Definition
description: |
Project-relative path to JSON file holding array of objects with `entity`, `path`, `keys`, and `encoding` [Optional] keys, as ascribed under [Files](#files):
Project-relative path to JSON file holding array of objects as described under [Files](#files) - with `entity`, `path`, `keys`, and other optional keys:

```json
[
Expand Down