Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-37021 [Java][arrow-jdbc] Pluggable getConsumer #37085

Merged
merged 8 commits into from
Aug 10, 2023

Conversation

aiguofer
Copy link
Contributor

@aiguofer aiguofer commented Aug 9, 2023

Rationale for this change

This was discussed in this thread. The getConsumer implementation depends on the implementation for getArrowTypeFromJdbcType, especially for timestamp objects. Since we can provide a different implementation for JdbcToArrowTypeConverter it follows we should be able to provide a different implementation for JdbcConsumerGetter

What changes are included in this PR?

Adds a way to configure an alternate JdbcToArrowUtils.getConsumer function through JdbcToArrowConfigBuilder.setJdbcConsumerGetter.

It also throws a more helpful exception from the default implementations when a provided type is not mapped.

Are these changes tested?

No, the default behavior remains unchanged.

@aiguofer aiguofer requested a review from lidavidm as a code owner August 9, 2023 06:31
@github-actions
Copy link

github-actions bot commented Aug 9, 2023

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

@aiguofer aiguofer changed the title [Java][arrow-jdbc] Pluggable getConsumer GH-37021 [Java][arrow-jdbc] Pluggable getConsumer Aug 9, 2023
@github-actions
Copy link

github-actions bot commented Aug 9, 2023

⚠️ GitHub issue #37021 has been automatically assigned in GitHub to PR creator.

Copy link
Member

@lidavidm lidavidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Aug 9, 2023
@github-actions github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 10, 2023
@lidavidm
Copy link
Member

Looks like a static analysis failure:

Error:  /arrow/java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/JdbcToArrowConfig.java:[201,8] error: [ChainingConstructorIgnoresParameter] The called constructor accepts a parameter with the same name and type as one of its caller's parameters, but its caller doesn't pass that parameter to it.  It's likely that it was intended to.
Error:      (see https://errorprone.info/bugpattern/ChainingConstructorIgnoresParameter)
Error:    Did you mean 'explicitTypesByColumnIndex,'?
Error:  -> [Help 1]

@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Aug 10, 2023
@github-actions github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 10, 2023
Copy link
Member

@lidavidm lidavidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Aug 10, 2023
@lidavidm
Copy link
Member

Hmm, it's not clear why, but CI is failing - does the test suite pass for you?

@aiguofer
Copy link
Contributor Author

Hmm no... it is hanging on TestFlightSql.testGetTablesResultFilteredWithSchema, trying to track it down.

@aiguofer
Copy link
Contributor Author

ahh tracked it down to an uncaught exception with unmapped JDBC types

@lidavidm lidavidm merged commit 0e4fa8a into apache:main Aug 10, 2023
13 of 15 checks passed
@lidavidm lidavidm removed the awaiting merge Awaiting merge label Aug 10, 2023
@conbench-apache-arrow
Copy link

After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 0e4fa8a.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about possible false positives for unstable benchmarks that are known to sometimes produce them.

loicalleyne pushed a commit to loicalleyne/arrow that referenced this pull request Nov 13, 2023
### Rationale for this change

This was discussed in [this thread](apache#37021 (comment)). The `getConsumer` implementation depends on the implementation for `getArrowTypeFromJdbcType`, especially for timestamp objects. Since we can provide a different implementation for `JdbcToArrowTypeConverter` it follows we should be able to provide a different implementation for `JdbcConsumerGetter`

### What changes are included in this PR?

Adds a way to configure an alternate `JdbcToArrowUtils.getConsumer` function through `JdbcToArrowConfigBuilder.setJdbcConsumerGetter`.

It also throws a more helpful exception from the default implementations when a provided type is not mapped.

### Are these changes tested?

No, the default behavior remains unchanged.
* Related: apache#37021
* Closes: apache#37021

Authored-by: Diego Fernandez <[email protected]>
Signed-off-by: David Li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Java][JDBC] Converting TIMESTAMP_WITH_TIMEZONE
2 participants