-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-37021 [Java][arrow-jdbc] Pluggable getConsumer #37085
Conversation
Thanks for opening a pull request! If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project. Then could you also rename the pull request title in the following format?
or
In the case of PARQUET issues on JIRA the title also supports:
See also: |
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/JdbcToArrowUtils.java
Outdated
Show resolved
Hide resolved
java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/JdbcToArrowConfig.java
Outdated
Show resolved
Hide resolved
java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/JdbcToArrowConfig.java
Outdated
Show resolved
Hide resolved
Looks like a static analysis failure:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Hmm, it's not clear why, but CI is failing - does the test suite pass for you? |
Hmm no... it is hanging on |
ahh tracked it down to an uncaught exception with unmapped JDBC types |
After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 0e4fa8a. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about possible false positives for unstable benchmarks that are known to sometimes produce them. |
### Rationale for this change This was discussed in [this thread](apache#37021 (comment)). The `getConsumer` implementation depends on the implementation for `getArrowTypeFromJdbcType`, especially for timestamp objects. Since we can provide a different implementation for `JdbcToArrowTypeConverter` it follows we should be able to provide a different implementation for `JdbcConsumerGetter` ### What changes are included in this PR? Adds a way to configure an alternate `JdbcToArrowUtils.getConsumer` function through `JdbcToArrowConfigBuilder.setJdbcConsumerGetter`. It also throws a more helpful exception from the default implementations when a provided type is not mapped. ### Are these changes tested? No, the default behavior remains unchanged. * Related: apache#37021 * Closes: apache#37021 Authored-by: Diego Fernandez <[email protected]> Signed-off-by: David Li <[email protected]>
Rationale for this change
This was discussed in this thread. The
getConsumer
implementation depends on the implementation forgetArrowTypeFromJdbcType
, especially for timestamp objects. Since we can provide a different implementation forJdbcToArrowTypeConverter
it follows we should be able to provide a different implementation forJdbcConsumerGetter
What changes are included in this PR?
Adds a way to configure an alternate
JdbcToArrowUtils.getConsumer
function throughJdbcToArrowConfigBuilder.setJdbcConsumerGetter
.It also throws a more helpful exception from the default implementations when a provided type is not mapped.
Are these changes tested?
No, the default behavior remains unchanged.