Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tidb2dw should only has dependency on shared storage #101

Open
Tracked by #100
zhangjinpeng87 opened this issue Mar 25, 2024 · 0 comments
Open
Tracked by #100

tidb2dw should only has dependency on shared storage #101

zhangjinpeng87 opened this issue Mar 25, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@zhangjinpeng87
Copy link
Collaborator

zhangjinpeng87 commented Mar 25, 2024 •
edited
Loading

Right now the tidb2dw will connect to upstream TiDB to fetch some information like schema and column default value which break the simple dependency principle. The tidb2dw service should only have dependcy on the shared object storage like S3 and GCS, in this way, there should not have upstream database permission issue and network dependency issues since upstream clusters might in different VPC networks.

tidb -> ticdc ----->
                    --------> S3/GCS ------->  
                                              -------> tidb2dw -------> Snowflake/Redshift/...
                    --------> S3/GCS ------->
tidb -> ticdc ----->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@zhangjinpeng87