Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: standard for building new connector #49

Closed
pnadolny13 opened this issue May 22, 2023 · 2 comments · Fixed by #52
Closed

docs: standard for building new connector #49

pnadolny13 opened this issue May 22, 2023 · 2 comments · Fixed by #52

Comments

@pnadolny13
Copy link
Contributor

pnadolny13 commented May 22, 2023

We should document the minimum standard we have for creating a new connector. This can act as a check list for us to know that a connector has completed initial development and should be made available to the community. It also acts as a way to communicate with external contributors or contractors what the expectation and best practices are for creating a new connector.

In #37 (comment) I created a list of common checklist steps, this could be a good starting place.

  • tagged release
  • integration tests pass in CI
  • branch protection on main
  • cookiecutter TODOs completed and removed
  • clean README
  • Added to meltano hub
  • Clearly document the feature support comparison of the new tap vs the default variant on the hub

Some other best practices that have been mentioned are:

  • When building from scratch always use the cookiecutter to make sure the newest set up and recommended scaffold is present. This is compared to starting with a clone of an existing SDK repo and modifying it for the new source.
  • Use the latest SDK version
  • tap.py settings should reflect what should be shown on the hub. All required, default, descriptions, etc. should be accurate in the tap.
  • populate the first section of the README using the auto generated --about --format=markdown output
  • General coding practices: https://handbook.meltano.com/engineering/dev-standards/
  • pyproject.toml package version and github/pypi release version should match
  • Verify that there are no extra unused dependencies in the pyproject.toml and all dev dependencies are configured as such
  • Catalogs should be supported properly to control output based on selection criteria
  • logging, comments, doc strings, function/class/variable naming, etc. should be reviewed for clarity and accuracy
  • All known issues and planned improvements should be documented as issues in the repository

@kgpayne @edgarrmondragon @tayloramurphy let me know what you think of these and if you have more. We can add them to the README or a contributors guide when we're ready.

@tayloramurphy
Copy link
Contributor

@pnadolny13 this is a great start - let's just get a PR up and ready, perhaps within MeltanoLabs?

The other item(s) I would add is:

  • validate license is added
  • naming convention of the connector itself

@edgarrmondragon
Copy link
Member

tests pass in CI

@pnadolny13 Should probably clarify this means integration tests

pnadolny13 added a commit that referenced this issue May 31, 2023
Closes #49

- cleanup the README and add a link out to the connector dev standards
guide
- add connector dev standards guide

Let me know if I missed any tips or if you think theres better
formatting or location for this.

---------

Co-authored-by: Edgar R. M. <[email protected]>
@github-project-automation github-project-automation bot moved this to Planned in Data Team May 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants