-
Notifications
You must be signed in to change notification settings - Fork 931
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support to specify client kerberosAuthType as fromTicketCache #5961
Conversation
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java
Outdated
Show resolved
Hide resolved
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #5961 +/- ##
============================================
- Coverage 61.27% 61.17% -0.11%
Complexity 23 23
============================================
Files 622 622
Lines 36882 36896 +14
Branches 5014 5014
============================================
- Hits 22599 22570 -29
- Misses 11854 11885 +31
- Partials 2429 2441 +12 ☔ View full report in Codecov by Sentry. |
35bcbee
to
c01a99e
Compare
…etCache # 🔍 Description The use case: - 1. the user code running in flink cluster and wrapped by transparent `ugi.doAs` - 2. the current UserGroupInfomation is the proxy user not login/real user - 3. user specify url with kyuubiClientTicketCache but does not work - 4. the proxy UserGroupInfomation is used and then throw GSS issue. ``` Caused by: org.apache.kyuubi.shade.org.apache.thrift.transport.TTransportException: GSS initiate failed at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.lambda$open$0(TSubjectTransport.java:47) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.open(TSubjectTransport.java:42) at org.apache.kyuubi.jdbc.hive.KyuubiConnection.openTransport(KyuubiConnection.java:458) at org.apache.kyuubi.jdbc.hive.KyuubiConnection.<init>(KyuubiConnection.java:207) ... 22 more ``` The root cause is that, for this case, the result of `isHadoopUserGroupInformationDoAs` is true. So, `isFromSubjectAuthMode` is true. In this pr, I want to specify the kerberosAuthType to fromTicketCache, and do not check `isHadoopUserGroupInformationDoAs`. After this pr, customer can specify `kerberosAuthType=fromTicketCache` to leverage ticket cache prefer than `ugi.doAs`. ## Issue References 🔗 This pull request fixes # ## Describe Your Solution 🔧 Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. ## Types of changes 🔖 - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request ⚰️ #### Behavior With This Pull Request 🎉 #### Related Unit Tests --- # Checklist 📝 - [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes #5961 from turboFei/warn_ticket_cache. Closes #5961 6e06500 [Fei Wang] exception c01a99e [Fei Wang] refine 098a37b [Fei Wang] do not infer fromSubject if fromTicketCache && ugi 042fa22 [Fei Wang] warn Authored-by: Fei Wang <[email protected]> Signed-off-by: Cheng Pan <[email protected]> (cherry picked from commit 54086b0) Signed-off-by: Cheng Pan <[email protected]>
# 🔍 Description Followup #5961 ``` scala> Files.exists(Paths.get(null)) <console>:14: error: ambiguous reference to overloaded definition, both method get in class Paths of type (x$1: java.net.URI)java.nio.file.Path and method get in class Paths of type (x$1: String, x$2: String*)java.nio.file.Path match argument types (Null) and expected result type java.nio.file.Path Files.exists(Paths.get(null)) ^ scala> Files.exists(Paths.get("")) res0: Boolean = true scala> ``` ## Issue References 🔗 This pull request fixes # ## Describe Your Solution 🔧 Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. ## Types of changes :bookmark: - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request :coffin: #### Behavior With This Pull Request :tada: #### Related Unit Tests --- # Checklist 📝 - [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes #5980 from turboFei/fix_npe. Closes #5961 99d4a16 [Fei Wang] fix npe Authored-by: Fei Wang <[email protected]> Signed-off-by: Cheng Pan <[email protected]>
# 🔍 Description Followup #5961 ``` scala> Files.exists(Paths.get(null)) <console>:14: error: ambiguous reference to overloaded definition, both method get in class Paths of type (x$1: java.net.URI)java.nio.file.Path and method get in class Paths of type (x$1: String, x$2: String*)java.nio.file.Path match argument types (Null) and expected result type java.nio.file.Path Files.exists(Paths.get(null)) ^ scala> Files.exists(Paths.get("")) res0: Boolean = true scala> ``` ## Issue References 🔗 This pull request fixes # ## Describe Your Solution 🔧 Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. ## Types of changes :bookmark: - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request :coffin: #### Behavior With This Pull Request :tada: #### Related Unit Tests --- # Checklist 📝 - [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes #5980 from turboFei/fix_npe. Closes #5961 99d4a16 [Fei Wang] fix npe Authored-by: Fei Wang <[email protected]> Signed-off-by: Cheng Pan <[email protected]> (cherry picked from commit cbe177b) Signed-off-by: Cheng Pan <[email protected]>
…omTicketCache # 🔍 Description The use case: - 1. the user code running in flink cluster and wrapped by transparent `ugi.doAs` - 2. the current UserGroupInfomation is the proxy user not login/real user - 3. user specify url with kyuubiClientTicketCache but does not work - 4. the proxy UserGroupInfomation is used and then throw GSS issue. ``` Caused by: org.apache.kyuubi.shade.org.apache.thrift.transport.TTransportException: GSS initiate failed at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.lambda$open$0(TSubjectTransport.java:47) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.open(TSubjectTransport.java:42) at org.apache.kyuubi.jdbc.hive.KyuubiConnection.openTransport(KyuubiConnection.java:458) at org.apache.kyuubi.jdbc.hive.KyuubiConnection.<init>(KyuubiConnection.java:207) ... 22 more ``` The root cause is that, for this case, the result of `isHadoopUserGroupInformationDoAs` is true. So, `isFromSubjectAuthMode` is true. In this pr, I want to specify the kerberosAuthType to fromTicketCache, and do not check `isHadoopUserGroupInformationDoAs`. After this pr, customer can specify `kerberosAuthType=fromTicketCache` to leverage ticket cache prefer than `ugi.doAs`. ## Issue References 🔗 This pull request fixes # ## Describe Your Solution 🔧 Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. ## Types of changes 🔖 - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request ⚰️ #### Behavior With This Pull Request 🎉 #### Related Unit Tests --- # Checklist 📝 - [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes apache#5961 from turboFei/warn_ticket_cache. Closes apache#5961 6e06500 [Fei Wang] exception c01a99e [Fei Wang] refine 098a37b [Fei Wang] do not infer fromSubject if fromTicketCache && ugi 042fa22 [Fei Wang] warn Authored-by: Fei Wang <[email protected]> Signed-off-by: Cheng Pan <[email protected]>
… exists # 🔍 Description Followup apache#5961 ``` scala> Files.exists(Paths.get(null)) <console>:14: error: ambiguous reference to overloaded definition, both method get in class Paths of type (x$1: java.net.URI)java.nio.file.Path and method get in class Paths of type (x$1: String, x$2: String*)java.nio.file.Path match argument types (Null) and expected result type java.nio.file.Path Files.exists(Paths.get(null)) ^ scala> Files.exists(Paths.get("")) res0: Boolean = true scala> ``` ## Issue References 🔗 This pull request fixes # ## Describe Your Solution 🔧 Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. ## Types of changes :bookmark: - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request :coffin: #### Behavior With This Pull Request :tada: #### Related Unit Tests --- # Checklist 📝 - [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes apache#5980 from turboFei/fix_npe. Closes apache#5961 99d4a16 [Fei Wang] fix npe Authored-by: Fei Wang <[email protected]> Signed-off-by: Cheng Pan <[email protected]>
…omTicketCache # 🔍 Description The use case: - 1. the user code running in flink cluster and wrapped by transparent `ugi.doAs` - 2. the current UserGroupInfomation is the proxy user not login/real user - 3. user specify url with kyuubiClientTicketCache but does not work - 4. the proxy UserGroupInfomation is used and then throw GSS issue. ``` Caused by: org.apache.kyuubi.shade.org.apache.thrift.transport.TTransportException: GSS initiate failed at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.lambda$open$0(TSubjectTransport.java:47) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.open(TSubjectTransport.java:42) at org.apache.kyuubi.jdbc.hive.KyuubiConnection.openTransport(KyuubiConnection.java:458) at org.apache.kyuubi.jdbc.hive.KyuubiConnection.<init>(KyuubiConnection.java:207) ... 22 more ``` The root cause is that, for this case, the result of `isHadoopUserGroupInformationDoAs` is true. So, `isFromSubjectAuthMode` is true. In this pr, I want to specify the kerberosAuthType to fromTicketCache, and do not check `isHadoopUserGroupInformationDoAs`. After this pr, customer can specify `kerberosAuthType=fromTicketCache` to leverage ticket cache prefer than `ugi.doAs`. ## Issue References 🔗 This pull request fixes # ## Describe Your Solution 🔧 Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. ## Types of changes 🔖 - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request ⚰️ #### Behavior With This Pull Request 🎉 #### Related Unit Tests --- # Checklist 📝 - [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes apache#5961 from turboFei/warn_ticket_cache. Closes apache#5961 6e06500 [Fei Wang] exception c01a99e [Fei Wang] refine 098a37b [Fei Wang] do not infer fromSubject if fromTicketCache && ugi 042fa22 [Fei Wang] warn Authored-by: Fei Wang <[email protected]> Signed-off-by: Cheng Pan <[email protected]>
… exists # 🔍 Description Followup apache#5961 ``` scala> Files.exists(Paths.get(null)) <console>:14: error: ambiguous reference to overloaded definition, both method get in class Paths of type (x$1: java.net.URI)java.nio.file.Path and method get in class Paths of type (x$1: String, x$2: String*)java.nio.file.Path match argument types (Null) and expected result type java.nio.file.Path Files.exists(Paths.get(null)) ^ scala> Files.exists(Paths.get("")) res0: Boolean = true scala> ``` ## Issue References 🔗 This pull request fixes # ## Describe Your Solution 🔧 Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. ## Types of changes :bookmark: - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request :coffin: #### Behavior With This Pull Request :tada: #### Related Unit Tests --- # Checklist 📝 - [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes apache#5980 from turboFei/fix_npe. Closes apache#5961 99d4a16 [Fei Wang] fix npe Authored-by: Fei Wang <[email protected]> Signed-off-by: Cheng Pan <[email protected]>
🔍 Description
The use case:
ugi.doAs
The root cause is that, for this case, the result of
isHadoopUserGroupInformationDoAs
is true.So,
isFromSubjectAuthMode
is true.In this pr, I want to specify the kerberosAuthType to fromTicketCache, and do not check
isHadoopUserGroupInformationDoAs
.After this pr, customer can specify
kerberosAuthType=fromTicketCache
to leverage ticket cache prefer thanugi.doAs
.Issue References 🔗
This pull request fixes #
Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Types of changes 🔖
Test Plan 🧪
Behavior Without This Pull Request ⚰️
Behavior With This Pull Request 🎉
Related Unit Tests
Checklist 📝
Be nice. Be informative.