Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DSIP-83][datasource-plugin] Add DolphinDB datasource #16962

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from
Open

[DSIP-83][datasource-plugin] Add DolphinDB datasource #16962

wants to merge 2 commits into from

Conversation

eco8848
Copy link

@eco8848 eco8848 commented Jan 15, 2025 •
edited by SbloodyS
Loading

Purpose of the pull request

  • This pull request adds DolphinDB datasource.

close #16818

Brief change log

  • Add DolphinDB datasource module.
  • Add DolphinDB documentation.
  • Add DolphinDB constant definitions, connection parameters, data source processors and other components.
  • Add DolphinDB JDBC driver dependency.

Verify this pull request

This change added tests and can be verified as follows:

  • Add DataSourceUtilsTest class to test the datasource utility methods.
  • Add DolphinDBDataSourceChannelFactoryTest class to verify the creation methods of the datasource channel factory.
  • Add DolphinDBDataSourceChannelTest class to validate the creation of the datasource client through the channel.
  • Add DolphinDBDataSourceProcessorTest class to test various functionalities of the datasource processor.

@boring-cyborg boring-cyborg
Copy link

boring-cyborg bot commented Jan 15, 2025

Thanks for opening this pull request! Please check out our contributing guidelines. (https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)

@github-actions github-actions bot added UI ui and front end related backend test document labels Jan 15, 2025
@eco8848
Copy link
Author

eco8848 commented Jan 15, 2025

Related DSIP #16818

@SbloodyS SbloodyS changed the title [Feature-15953][datasource-plugin] Add DolphinDB datasource [DSIP-83][datasource-plugin] Add DolphinDB datasource Jan 15, 2025
@SbloodyS SbloodyS added this to the 3.3.0 milestone Jan 15, 2025
@SbloodyS SbloodyS added the DSIP label Jan 15, 2025
github-advanced-security[bot]
github-advanced-security bot found potential problems Jan 15, 2025
View reviewed changes
.../apache/dolphinscheduler/plugin/datasource/dolphindb/param/DolphinDBDataSourceProcessor.java

String[] hostSeperator = connectionParams.getAddress().split(Constants.DOUBLE_SLASH);
String[] hostPortArray = hostSeperator[hostSeperator.length - 1].split(Constants.COMMA);
dataSourceParamDTO.setPort(Integer.parseInt(hostPortArray[0].split(Constants.COLON)[1]));

Check notice

Code scanning / CodeQL

Missing catch of NumberFormatException Note

Potential uncaught 'java.lang.NumberFormatException'.
SbloodyS
SbloodyS reviewed Jan 22, 2025
View reviewed changes
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add dolphinscheduler-datasource-dolphindb to https://github.com/apache/dolphinscheduler/blob/dev/config/plugins_config

dolphinscheduler-bom/pom.xml
@@ -57,6 +57,7 @@
<cron-utils.version>9.1.6</cron-utils.version>
<h2.version>2.2.220</h2.version>
<mysql-connector.version>8.0.33</mysql-connector.version>
<dolphindb-jdbc.version>3.00.0.1</dolphindb-jdbc.version>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use the latest version 3.00.0.2?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already added dolphinscheduler-datasource-dolphindb to plugins_config. The jdbc version used during development and testing is 3.00.0.1.

@davidzollo davidzollo added the first time contributor First-time contributor label Jan 22, 2025
@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
18 New issues
0 Accepted issues

Measures
0 Security Hotspots
64.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SbloodyS SbloodyS SbloodyS left review comments

@EricGao888 EricGao888 Awaiting requested review from EricGao888 EricGao888 is a code owner

@songjianet songjianet Awaiting requested review from songjianet songjianet is a code owner

@caishunfeng caishunfeng Awaiting requested review from caishunfeng caishunfeng is a code owner

@ruanwenjun ruanwenjun Awaiting requested review from ruanwenjun ruanwenjun is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees

@eco8848 eco8848

Labels
backend document DSIP first time contributor First-time contributor test UI ui and front end related
Projects
None yet
Milestone
3.3.0
Development

Successfully merging this pull request may close these issues.

[DSIP-83][Feature] Add DolphinDB datasource support to DolphinScheduler
3 participants
@eco8848 @SbloodyS @davidzollo